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Editorial 



This report reflects research and development of the Department of Applied Education, 

Division of Instrumental Science and Technology of the University of Twente in the 

Netherlands and Andersen Consulting - ECC, also located at the campus of the University of 

Twente, and part of the world wide consultancy firm Andersen Consulting. 

Both groups carry out work on instructional technology and are exploring concepts of 

visualisation and the use of multi media. 

This report addresses three aspects of instructional visualisation: 

- concepts in instructional visualisation 

- design and development of multi media applications 

- multi media technology 

Ivan Stanchev, Associate Professor ,Faculty of Educational Science and Technology, 
University of Twente. 

Jan Schoenmaker, Associate Partner, Andersen Consulting - ECC. 
Enschede, January 1994 



Visualisation and Effective Instruction 



Jef Moonen 



Abstract 

In this paper the potential relationship between visualisation and effective instruction is 
being reviewed. Reference is made to the AIME concept introduced by Salomon. It is 
concluded that research in this area should taken into account aspects of costs, cost- 
effectiveness and motivation of the learner. 



Introduction 

Improving the quality of learning and instruction is a continuing objective of education 
and training. 'Quality' however, can be described in many respects. One perspective is to 
focus on the effectiveness of the learning material to support the instruction. 

As the traditional classroom and training situation is being expanded with technological 
features, such as video, computer and multimedia there is considerable interest nowadays 
relative to the possibilities of graphical user interfaces of electronic learning material in 
relation to visual communication. Presenting, transmitting and processing information in 
visual, non-textual form is what is meant when we speak of visual communication. Non- 
textual symbols, pictures, graphs, images and so on that convey information are called 
visuals. 

The importance of visualisation in electronic learning material is reflected in many 
authoring systems as they offer more and more-varied possibilities to include visuals into 
the courseware they produce. However, it remains difficult to derive guidelines with 
respect to the integration of visual communication in courseware design. Improvement of 
courseware design methodology is therefore necessary; in particular, by investigating the 
relationships between type of courseware and the functions of visualisation. 

In that context there is a need for further development of theoretical principles of the 
visual communication, especially with respect to the following questions: 

- In what circumstances does the mind prefer the more complex, dynamic, lifelike image 
to the seemingly simpler verbal one? 

- How do non-textual symbols, graphics, still pictures and moving pictures contribute to 
visual communication? 

- If visual literacy is the ability to understand, think, and create non-verbally, how should 
visual literacy be learned and trained? 

- In what way will visual learning be of value in determining the (cost)-effectiveness of 
the learning process? 
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A special aspect is the inclusion of visual techniques in tools to handle non-verbal 
communication, and in tools supporting the courseware design process. The latter is a 
logical consequence of what is going on in computer science itself, whereby two new 
styles of human-computer interaction have become popular: 'programming through visual 
environments in which graphical elements play prominent roles alongside text, and iconic 
environments, in which users interact with the machine primarily by defining, pointing at, 
juxtaposing, overlapping and otherwise manipulating postage-stamp size images commonly 
referred to as icons' (Glinert, 1990, p. 1). 

Answers to the *bove about visual communication should help to better identify the 
advantages and disadvantages for visualisation of information in courseware and 
courseware design, leading to a new set of questions: 

- What kind of tools improve the process of visual commuiiication in teaching and 
training? 

- What kind of tools improve the process of courseware design? 

- What are the aspects and the factors that determine the break-even-point of using 
visualisation in terms of cost-effectiveness of courseware design, development and 
implementation? 

- How does visualisation improve the cultural portability of electronic learning material? 

In addition, the evolution of human-computer interface technology will significantly 
change in the next decade, given the growing international research interest in virtual 
reality (Earnshaw, Gigante, Jones, 1993; Kalawsky, 1993). As with any new technology, 
there are pedagogical questions to be addressed as methods of instruction are extended 
from being dominantly text-based to multi-sensory based. Additional questions such as the 
following are also important: 

- How is learning in a "virtual reality" different from that of a traditional educational 
environment? 

- How are learning styles enhanced or changed by virtual reality? 

- What kinds of research will be needed to assist instructional designers in developing 
effective virtual-reality learning environments? 

Results from Research 

Because of the evolving technology, visual messages in different forms can now be much 
easier presented than in the past. However, there are no consistent research results to 
support the hypothesis that by using visuals the effectiveness of what is being delivered is 
increased. 

Print material and visuals 

In relation to print material Fleming and Levie (1993) indicate for instance that 'although 
the results of research on the effects of using graphics are neither consistent nor 
compelling, most authorities and professionals are convinced that in many circumstances 
graphics help readers (especially poor readers) to use and understand instructional text' (p. 
41), and 'although research has failed to demonstrate conclusively that using pictures in 
courseware (in this context text-based material) is related to motivation, their instincts tell 
many designers that good pictures really do motivate learners' (p. 46). In the context of 
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message tieiigi? variables and with respect to the perception of pictures, Fleming and Levie 
present eight design guidelines. Their first and main guideline is that 'pictures are usually 
more memorable than words, and are thus useful when information has to be remembered* 
(p. 86). However, their second guideline is that 'pictures play many roles in instruction. It 
is therefore necessary to know precisely what a picture's function is intended to be before 
it is designed* (p. 86). 

Graphic design principels and interactive systems 

In their discussion of the design of effective visual presentation Baecker and Buxton 
(1987) report that the application of graphic design principles to the medium of computer 
displays and interactive systems is not a trivial process. They suggest that 'given the 
difficulty of the problem, good progress will probably be achieved through the multi- 
disciplinary collaboration of the technologist "telling us what is possible", the psychologist 
"telling us what not to do", and the designer "suggesting what to do" (p. 300-301)'. 

Cognitive psychology and user-interface design 

Gardiner & Christie (1987) investigate relevant areas of cognitive psychology in relation 
to user-interface design, and summarize their findings in 162 design guidelines. They 
discuss the relationship of the use of visual material and its impact on memory and 
conclude, based upon the dual-coding theory of Paivio, that 'our ability to remember the 
appearance of novel, unorganized visual patterns over the short-term is extremely limited. 
On the other hand, our ability to remember the identity of a concrete item is better when it 
is presented as a picture than as a printed word' (p. 128). And they further conclude that 
'ideally interfaces should be designed to be flexible enough to allow the user to vary the 
amount of information they have to deal with at any time, and, conversely, should not 
allow users to "get away" with inadequate amounts of processing where the consequences 
of poor recall will be dire' (p. 157). More generally Gardner and Christie summarize 
research findings into two principles: 

(a) 'The probability of recalling an item increases as a direct function of the depth 
of processing at which the item was encoded when punt into memory. Depth of 
processing increases with the requirement to consider the meaning of an item and 
its relation to other items'; 

(b) 'The probability of recalling an item increases as a direct function of the 
elaborateness of processing associated with its encoding into memory. 
Elaborateness of processing increases with the richness of the context information 
present at the time of the storage' (p. 159). 

These principles lead to the following design guideline: 'Items which must be recalled 
from memory some time after being removed from the screen should be surrounded with 
supporting information to which they are related, and should be immersed in a visually 
rich environment' (Design Guideline 125, p. 268). 

Guidelines for the design of visuals for information 

Petterson (1989) presents practical guidelines for the design of visuals for information. He 
stresses that image variables should be considered in relation to content, graphic execution, 
context and format. In the design and production of visuals for instruction, pictures must 
obviously contain the information they are intented to convey and must be relevant to the 
situation. However, Petterson reminds us of the fact that understanding a message in a 
visual are different in various cultures as well as in different socia-economic groups. 
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Lanzing (1993) has also done research into this area. 
Relationship between verbal and visual messages 

In addition, effective human-machine interaction requires an understanding of the 
relationships between verbal and visual messages, between characteristics of the learning 
material and the instructional process, the instructional approach and learning styles. 
Verleur (1993) presents an interesting summary of research results in this context. 
Generally speaking, research results indicate that when audio and visuals are presented 
simultaneously, the visually presented information will be dominant (Jaspers, 1991). 
Another conclusion is that presentations which are focussing on two senses or using two 
channels including an iconic presentation and a linguistic approach (text or audio) are 
superior to a presentation using only one channel. The most powerful result of using 
multi-channel presentation forms are their positive impact on the motivation of the learner. 

Empirical research 

Although many guidelines appear in the literature about how to incorporate visuals into 
learning materials, most of such guidelines are based upon theoretically driven research. 
Such research tends to seek out simple paradigms which appear to focus successfully on a 
single psychological process. It should be underlined that empirical research is necessary, 
as factors such as individual skills and knowledge and the use of multi-sensoi7 channels 
can make significant differences to performance and motivation. In other words, general 
conclusions about the impact of visualisation on the effectiveness of learning material 
remain to be supported. 

In particular a main focus for research are the circumstances under which visualisation 
realises a positive contribution to the quality of the instruction/learning process. 

AIME: The Amount of Invested Mental Effort 

An interesting approach with respect to the impact of television on learning was developed 
by Salomon (1984). 

When the structure of a communication is ^ore or less congruent with one's past 
experience, that information is processed rather mindlessly. When on the contrary, 
nonautomatic and effortful mental elaborations are necessary to capture the information, 
these lead to both depth and mindfulness. A measure to capture such efforts is called by 
Salomon "the amount of invested mental effort" (AIME). 

Two other concepts affect the AIME executed by a person. The first is the learners* 
perceived task characteristics (PDC) of the task. The more demanding a PDC is, the more 
AIME will be expended. The second is the learners* perceived self-efficacy (PSE). 
According to Salomon: The more efficacious learners perceived themselves to be, the 
more they are likely to invest sustained effort in a task and persist in doing so* (p. 649) . 
Assuming that thorough and mindful coverage of information are characteristics of better 
quality of the instruction/learning process, Salomon argues that such characteristics will be 
obtained when the instructional process stimulates an increase of AIME. 

Trie main issue with respect to visualisation therefore is not to investigate technical 
possibilities of new media in order to 'maximise* the portion of visualisation within an 
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instructional program. The main issue is to match the visualisation possibilities of n 
media with the characteristics of the learners, in order to maximize their amount of 
invested mental effort. 



Do Media Influence Learning? 

As we talk about visualisation, it is not the medium but its presentation formats that are of 
importance. 

In order to better distinguish different aspects of media, Kozma (1991) defines the media 
in terms of their technology, their symbol system, and their processing capabilities. 
Most common is to talk about media in terms of their technology: the mechanical and 
electronic aspects that determine their functioning and physical features. There are TVs, 
VCRs, CD-ROM players, slide projectors, etc. 

The most interesting features of a medium however are its possibilities to present a range 
of symbols (for instance text, audio, visual), and its processing capabilities (for instance* 
interactivity, search capabilities). 

Questioning if media through visualisation will contribute to maximizing AIME clearly 
relates to the range of symbol systems the medium can present, and to its processing 
capabilities. 

In Paragraph 2 it was emphasided that empirical research should be done in order to 
determine in practice the imoact of visualisation on the effectiveness of the 
teaching/learning process. Given the different characteristics of media, this emphasis 
should not concentrate on the technical possibilities of new and future media, or on the 
comparison of one technical facility versus another. On the contrary, research should be 
planned in relation to new symbol systems and/ or new processing capabilities represented 
in new and future media. Unless a new medium creates such a new symbol system or a 
new processing capability, a new medium has to be considered as belonging to a specific 
class of media capable of a certain kind of presentation and information transfer. 

In that respect, for instance, a product presented through CD-I is not essentially different 
from a product presented through a combination of a computer program controlling a 
interactive videodisc. In terms of instructional effectiveness there seems to be no reason 
why there should be a difference between these two media. However, there could be a 
significant different in terms of their cost-effectiveness. 



Costs, Cost-Effectiveness and Motivation 

Different media are not alike in terms of costs involved: costs with respect to 
procurement, and costs with respect to the development of learning materials. Neither are 
they alike in terms of their potential to motivate the learner, or put it in a broader context, 
to stimulate the AIME of the learner. 

When the expected instructional effect for two media are more or less comparable, then 
the cheapest medium is the most cost-effective. 
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Given the wealth of traditional and electronic media that are now available, and given the 
fact that many of these media have comparable features with respect to the symbol 
systems they can represent, and comparable power in terms of their processing 
capabilities, the question of the cost-effectiveness of a medium is becoming most relevant. 
Analysis of data in the context of cost-effectiveness very often lead? to the understanding 
that potential effects such as the instructional impact of a raise in motivation or change in 
attitude of the learners are not (well) represented in the measurement oi the performance, 
while enlarged motivation of the learners seems to be the most apparent effect. 

Conclusion 

Research with respect to visualisation and media should be concentrated on investigating 
relationships between symbol systems— in particular visualisation-and processing features, 
learners characteristics and learning styles. In addition researching the instructional value 
of visualisation, both aspects-cost-effectiveness and motivation-should be taken into 
account. 
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Principles and R&D Aspects in Instructional Visualisation 



Ivan Stanchev 



Communication - a model of the process 

How can we be clear and comprehensive in summarising the communication process in a 
single model ? We can't. So many complexities apply to human communication that we 
must settle for something less than completeness and clarity. But we can summarise some 
of the most significant aspects of the process, and we can sharpen our awareness of what 
goes on when we attempt to send and receive messages. 

Contemporary scholars have offered dozens of definitions of communication, and it would 
be an-oganf and unprofessional to come up with "a definition to end all definitions"; 
however, we offer this working definition of effective communication : 

Effective communication occurs when a sender transmits a message and a receiver 
responds to the message in a manner which satisfies the sender. 

Note a few implication of this definition: 

- It focuses on applied communication. This brief definition deals with getting results 
through communicating effectively in social organisations; it does not deal with theories, 
experiments or philosophies which go beyond that goal. 

- It accepts the satisfying concept. In social organisations, as in almost all human 
situations, we often accept what is less than perfect. As long as a situation becomes 
productive or meets our basic goals, we usually settle for it. Teachers and instructors tend 
to settle for a productive understanding, rather than pressing for clarification of every 
historical, linguistic, philosophical, or technical aspect of a communication. The time and 
energy involved in reaching a "perfect" understanding (assuming for the moment that it is 
possible) would be too costly. 

- It assumes a feedback loop. In order to realise satisfaction, parties must get some sort of 
confirming response. This can be verbal or non verbal, written or oral. It can occur 
through observation, evaluation systems, dialogue, and so on, but somehow the sender 
must experience verification of an adequate understanding of the intent of the message. 

Of course, in the process of giving feedback, the receiver becomes a sender and the 
original sender becomes a receiver. In addition, the original sender may modify a 
message in the light of feedback. But at each stage, our definition assumes sufficient 
response to confirm adequate understanding. 

The key word in the definition is the term "message". A "message" is a pattern of signs 
(words and pictures) produced for the purpose of modifying the cognitive, affective, or 
psycho-motor behaviour of one or more persons. The term does not imply any particular 
medium or vehicle of communication. 
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When we pay attention to a stimulus, we tend to give it an interpretation, whether the 
stimulus is verbal or non verbal. We base our interpretation on such factors as our 
general backgrounds, our loyalties, our vocabularies, our education, our expectations, our 
personal interests, our values and our prejudices. Because of such factors, distortions 
often occur as messages filter through channels. But by being aware of such factors, we 
can work toward encoding and decoding more effectively, and toward reducing "noise" in 
the communication process. 

Visualisation and perception 

In a letter to Jacques Hadamard, a friend of him, Albert Einstein speaking about himself 
wrote that words in their written or oral form did not play a significant role in his way of 
thinking and that to him the basic elements of thought were definite signs and 
symbols(Einstein, 1953). 

Mastering of techniques for information visualisation and the ability to present this 
information as an explicit and simple drawing have great importance in many spheres of 
human activity - research, design and construction activities, books and printed matter 
layout, etc. Visualisation of information takes a particularly important place in educational 
systems since psychologists have proved that visual perceptions play a most prominent 
part in information memorising and reproduction and that memory is the result of the 
received perceptions which according to some theorists can - apportioned in the 
following way: 

- 75 % visual perceptions; 

- 15 % aural impressions; 

- 6 % tactile perceptions; 

- 3 % sense of smell, and 

- 3 % taste. 

All that is of great importance when interactive courseware is being thought out and 
designed since the computer screen is the main source of information in the user's 
communication with a program. If the program's creators have not made it visual enough 
it could be difficult for the trainee to interact with it without using additional information 
from other sources (for example, from the teacher or a guide for work with the program). 
In contrast, the design of visual objects demonstrating the program's potential and content 
can improve the ease of comprehension and sound memorisation of the information 
contained in the program. 

The process of artistic designing (visual design) is very close to the theory and practice of 
fine arts - it is subjected to the same principle* with regard to the means of the 
composition (symmetry and asymmetry, metrics and rhythm, contrast and nuances, 
proportions, scale, colour, texture, etc.). The observation of these principles increases the 
strength of the impact of the information apprehended - a fact well known and used in the 
fields of ergonomy, aesthetics and design. On the basis of these principles a number of 
important and fundamental principles in the visualisation of information can be 
formulated: 
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The principle of conciseness. The graphic means for information presentation 
should contain only such elements which are indispensable for conveying essential 
information to the viewer and this information must be comprehensible for it represents 
the visual accent of the basic composition elements. It is useless to try to draw attention 
to the most important characteristic features of a situation if they are surrounded by 
superfluous visual distracters having no bearing to them and hindering the apprehension of 
the essential (see Fig. 1). 




The principle of generalisation and unificatio n. The basic forms for graphic 
presentation of information must not be needlessly broken to pieces or include elements 
designating superfluous details from the standpoint of the presented information. This form 
must be rationally generalised and the symbols denoting the same objects and phenomena 
must be unified, i.e., they must have a unified graphic solution (see Fig. 2). 



Figure 2 

The principle of accent on basic notional elements . In visualising information the 
elements most essential for information comprehension by the viewer should be set apart 
in size, form and colour. In some cases it is even possible to allow a conscious breaking- 
down of the proportions of symbols' sizes with respect to the real objects depicted by 
them (sec Fig. 3). 
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Figure 3 



The principle of autonomy . The graphic presentation of information concerning 
autonomous, independent objects or phenomena should be set apart and clearly 
differentiated from the remaining components of the image. In practice it means that the 
decomposition of the complex graphic information into separate simple images facilitates 
considerably its perception and comprehension (see Fig. 4). 




Figure 4 



The structural principle . Every autonomous component taking some central, crucial 
position in the graphic expression must have an exact and easily remembered structure 
differentiated from the other components. This structure, as well as the structure of the 
whole graphic image must conform to the logic structure of the real object or 
phenomenon's decomposition (see Fig. 5). 
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Figure 5 



The phasic nature principle. The behaviour of dynamic objects and phenomena 
presented through the means of information visualisation should be decomposed with 
respect to time and space. It helps not only to understand and acquire the mechanism of 
this behaviour but provides a procedural logic which should be presented with the 
assistance of various graphic r leans (see Fig. 6). 




Figure 6 
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The principle of use of fixed associations and stereotypes . In creating graphic 
forms for information presentation the stable, habitual associations between the symbols 
and the objects and phenomena designated with them must be recognised as well as the 
stereotypical reactions of the viewer to definite symbols and signals. It means that where it 
is possible one must not use abstract, conditional signs but symbols which lead through 
association to the respective objects and phenomena. On the other hand, however, one 
must bear in mind that the naturalist, detailed presentation of the outer appearance of the 
objects keeps the viewer's thought on the outer similarity with the object and impedes the 
realisation of .the more essential, from the point of view of the presented information, 
characteristics and regularities (see Fig. 7). 



Die Teile eines Computers 




Figure 7 
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How can abstract principles be applied to concrete instructional problems? One means of 
facilitating this transfer is to state both propositions (both the principle and the problem) in 
parallel forms. This form can be generalised as: "What conditions lead to what results?". 
In (Fleming and Levie, 1978, pp. 15-18) is given a excellent example for applying this 
approach using the basic principles of perception for some general guidelines for the 
instructional designer. 



Basics of visual communication 

Specialists in the field of visual communication maintain that a drawing is worth 
thousands of words and this is really so provided the drawing is good. The difference 
between words and a drawing is that with the assistance of words we narrate while a 
drawing shows. However, in order to achieve the desired result the possibilities of the 
graphic language must be mastered. If the meaning of a text is revealed through words, 
the illustration "speaks" in the language of forms. The communicative possibilities of the 
graphic language can be understood if we analyse the goals of visual communication, i.e., 
if we try to answer the questions What? How? How much? and Where? with the help of 
graphic expression (Fig. 8): 

- What does the object represent? What is it? - it is a question which refers above 
all to the outer appearance of the object, its physical structure or the logical relation of its 
parts to the whole. Most often questions refer to the visual perception of the object's 
properties, but they can also refer to such properties which are invisible under ordinary 
circumstances and also to definite abstract notions connected with the organisation of the 
object itself. 

- How does one or another object operate? This is a question which concerns its 
physical movement, the logic's of functioning of its separate parts within the frames of the 
whole as well as the process of functioning of the whole object as a sequence of 
interrelated conditions. In these cases the demonstration of actions in static images 
requires the use of symbolic forms displaying movement. It applies with equal force both 
to real physical objects and phenomena and to ideal models presenting an abstraction of 
reality. 

- How much refers to the quantitative characteristics of the object, i.e. to the 
physical dimensions of the object, the quantitative relationships between its components 
and elements as well as to their correlation to the whole, the tendency of alteration of the 
physical characteristics, etc. The quantitative analysis of the object and its image involve 
compulsory a process of abstraction. 

- Where is a given object located or a given phenomenon take place? This is a 
question which refers to the place of the object, its location in space, its position with 
respect to other objects. Very often the differr ces between these three characteristics are 
difficult to detect but the basic questions relevant to them can be differentiated according 
to their functional meaning. 
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Visual language has like any other language a syntax's, semantics and pragmatics. 
The lowest level elements of forms, comparable to the characters of verbal language, are 
well known from geometry - point which has no dimensions of its own but shows a 
place, location and position, then line as a one-dimensional image. On the intermediate 
level, comparable to words, configurations of the lower level element get a basic meaning. 

\9 



On the high level the "words" of visual language combine to form expressions. Through 
line we most often show direction, stretch or movement. As an element of the word stock 
line can be used also to depict trajectories and routes, to designate boundaries and 
divisions. The regular and irregular geometric figures as well as the plane forms are a 
two-dimensional image of some type. Space occupied by them coincides with a definite 
area of the drawing. In this sense the figure as an element from the dictionary of the 
graphic language is used to designate contours, area, outline, frame, etc.(see Fig. 9). 

The definite hue, i.e. a relationship between the degrees of black and white as well as the 
colours, affords us a possibility to distinguish between separate elements of the object or 
to accentuate on the basis of contrast. Smooth transition and use of light shades allow us 
to describe volumetric forms and perspective changes (see Fig. 10). 

The texture of the graphic image is in fact the surface structure of the drawing and here 
we can use both abstract and symbolic and descriptive textures. In practice, they denote 
properties of the physical surfaces of the object and are usually used with other elements 
of the form - most often they are combined with colours. 
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Graphic language together with its elements is not created for its own sake - the form the 
space and the visual interaction are means for presentation of definite ideas, objects and 
phenomena. That is why visualisation always begins by setting some definite goal and 
then a content corresponding to the goal is chosen which is translated into the language of 
visual forms. It is afterwards that the drawing is constructed and in it the visual language 
is wholly used to develop and specify the visual image and its transformation into a visual 
model. That means that the conception or the visual idea, the artistic design and the 
graphic execution are the three main stages in die process of information visualisation. 

What are the practical conclusions that we should drawn when visualising the information 
m interactive programs? Above all the conclusions are relevant to the graphic design of 
the separate "screens" presenting a basic element in the process of human-computer 
interactive interaction. The development of computers, and personal computers in 
particular, the extension of their possibilities to work in the graphic mode, eliminates the 
difference between artistic design by means of paper, pencil and coloured pencils and 
design with the assistance of specialised computer packages of applied programs for 
information visualisation. 

The computer offers something more - while in the classic visualisation of information it 
is impossible to apply the animation approach with computers as a result of their quick 
operation and the consecutive deposit of images perceived by the human eye as a 
continuos alteration of the image we gain a number of additional possibilities for visual 
impact. 



Advantages and disadvantages of visualisation in education and training 

To be able to use visualisation to its maximum potential we need to understand what 
characteristics of visualisation cause it to be advantageous. In this way we can better 
estimate when visualisation will be the most beneficial. 



Psychological and instructional aspects 

The acceptance of individual differences leads to three possible approaches. The first 
approach is to make understanding as easy as possible for each individual by supporting 
both text and visual modes of presentation. The second approach is to present only the 
mode that best fits the individual student. Thirdly one can argue that everyone should be 
proficient in reading both text and visuals, so making it too simple would not stimulate the 
reader/viewer to train his abilities. We guess the solution to this dilemma lies in the nature 
of the instruction. The question whether or not to teach visual literacy is strongly related 
to questions of teaching common knowledge, it does not explicitly belong in the school 
curriculum but it is expected from good general education. Training and reschooling of 
adults has less general and more specific goals and the designer of learning materials in 
this area may therefore try to optimise learning outcome by neglecting such goals as visual 
literacy and optimise the learning by teaching in the mode most suited for the individual 
learner. 

A disadvantage of visualisation of the learning content is that many times the link between 
the symbol and its meaning are more or less arbitrary, based on associations of the mind. 



Because of individual di.crences one can never be certain someone else will have the 
same associative link and will therefore be able to correctly interpret the symbols used. 
On the other hand comprehension can be facilitated by adding context to the symbols. A 
lonely "trash can" icon on a computer screen could mean many things (time to put your 
garbage can on the sidewalk, what you have just typed is trash, etc.), adding it to the rest 
of the desktop concept (icons for disks and files, the windows and menus) could easily 
lead to the conclusion that the trash can can be used to throw away files and discs. 

Several instructional theories focus on the dilemma of either working from abstract to 
concrete or the opposite. Romiszowski (1981) describes for instance Bruner's classification 
of three levels of representation; - inactive, iconic and symbolic, advancing from 
concrete to abstract (p. 173) - and Landa's concern about the rule/example sequence 
(RUL-Eu vs. EG-RUL) (p. 179). Examples are usually used to present information 
realistically, but text is an abstraction itself. To make the best use of examples they should 
be as realistic as possible. An obvious way to do this is to make use of visualisation. 
Metaphors are used when a topic has no intrinsic realism. To make the topic more 
accessible an analogy is found between the topic and a less abstract phenomenon. The 
metaphor may be visualised and therefore given more realism. In terms of Bruner this will 
enable students to gain iconic experience with symbolic topics. Direct manipulation 
(Shneiderman, 1990) of abstract information is possible by visualisation of the metaphor. 
A major difference between text and visuals from a psychological point of view is the 
difference between the sequential nature of text and the parallel nature of visuals. For text 
a predefined direction of processing the chain of characters and words makes it a 
sequential mode of presentation. For visuals there is no predefined direction of reading, 
therefore it is possible to display simple items in such a wsy that it is easy to view the 
items 'at the same time'. 

The motivational aspect of visualisation is clear (Keller, 1983). Programs that look nice 
are fun to work with. Of course one runs the risk of distracting and therefore decreasing 
the learning effects, but in general the more motivated the learner is the better he learns. 
Although motivation from the niceness of visuals is extrinsic, and less beneficial than 
intrinsic motivation, the general attitudes towards the learning task, towards the program 
and towards computers are positively influenced by such feelings as the comfort of 
working with the program and the perceived locus of control (Hartley & Lovell, 1984). 

Cross-cultural aspects 

Since the beginning of written language people have realised that communication using the 
arbitrary rules of language is not equally understandable for all. The chance of not 
understanding the associative relations between symbol and meaning increases when 
cultural differences between sender and receiver become larger. In general one can say 
that the less assumptions are made about the rules of communication, the greater the 
probability of understanding. Communication by text requires language, and language is 
from this point of view nothing less than an enormous amount of communication rules. By 
using text the sender assumes the receiver will be aware of these rules and will know how 
to apply them (quickly enough) to process the message. Language is part of culture; 
therefore often cross-language will also mean cross-cultural. Cross-cultural communication 
by text is therefore a problem unless sender and receiver can agree to use one set of rules 
(one language), provided they possess knowledge of a common set of rules (speak a 
common language). 
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Pictograms are iconic symbols. Using graphical representations one can create an icon 
language, which is again a set of rules, bul the associative link between symbol and 
meaning should make the icon language more culturally independent than text. Examples 
are the pictographic language for traffic regulations or the icon language for organisational 
and functional focusing of areas and facilities in communication and meeting environments 
(airports, railway stations, conference centres, etc.;. The ui^cussion by Ossner (1988, pp. 
8-11) follows the same lines of reasoning and reaches the same conclusions. 
To increase the portability and effectiveness of educational software we have to investigate 
and to apply the full range of cross-cultural advantages of rising visualisation (Lanzing, 
1991). All these advantages have to be transformed into fricticaJ principles and guidelines 
for software engineers and instructional designers involved in the development and 
implementation of educational software. 

There is however one less positive remark about the advantages of visualisation. The 
possibilities of visuals to express verbal information detailed enough to communicate 
whatever you would want to communicate are limited. 

What we have been trying to communicate using pictograms has so far been relatively 
easy to say; "go there", "exit over here", "no smoking allowed", etc To express more 
complicated things using graphical communication might not make understanding easier, 
but at least everybody is equally unfamiliar with the symbols used, which could by itself 
increase mutual understanding. 

Technical aspects 

Every computer is well equipped to display characters, but only a few of them offer 
enough built-in aids to make the programming of an interactive system un a graphical 
screen easy. Also the absence of one standard graphics adapter requires the programmer of 
graphical software to inves* great efforts to make the courseware run on most computers. 
To overcome this problem one needs to use either an authoring system which supports 
graphical interaction or a library with adequate graphical functions or objects. 
Without the right tools the development of a program that uses visualisation will take 
considerably more time and effort and therefore more financial means. Depending on the 
goals, the available tools and experience using them, the difference between the production 
of courseware in text and in graphical mode will decrease. In the end it might even 
become cheaper to produce graphical courseware due to improved authoring tools. 
The most important limitation of the design of graphical software is the skill needed to 
produce acceptable graphics. To guarantee an acceptable standard of graphical design one 
really needs a professional graphical designer to make it look professional. No student can 
be fooled to work with courseware that looks amateuristic; however good the concept, the 
instructional design and programming may be, users look at the screen and if they see 
amateur graphics they will not be confident about the quality of the rest of the program. 
Of course this will significantly increase the costs of courseware production. 
Several techniques for visualisation in courseware engineering are at our disposal. 
Multimedia techniques combine the advantages of normal video (quality moving images, 
good production methods) with the interactivity of computers. But the application of these 
methods still require special devices, installed both with the designer and the user of the 
courseware. This causes investments which are presently too high for most educational 
institutions. 

Commercial organisations are more and more using such multimedia techniques for 
company training, where the costs of such techniques are low compared to personnel 



costs. This increased usage will probaK iuse the prices of technical support for 
visualisation to decrease a little, but w«„ still not very optimistic of the chances of the 
technology for handling these techniques being afforded in schools in the near future. 

Graphical techniques use the computer to display either prepared images stored on a 
computer-readable external medium or images instantly generated by the computer. 
Because it requires no extra equipment, these digital approaches are usually the cheaper 
techniques. Therefore these techniques will be used more often. But apart from these 
economical aspects, digital graphical methods for visualisation are also superior in 
versatility. Since the image is fully digital the computer can do all sorts of things with it, 
ranging from adding arrows and marks to image processing. 

For a long time the quality of computer graphics has been rather poor and with a low 
degree of realism, due to both display technology and to the difficulty and labour 
intensiveness of entering pictures into the computer. The use of scanning technique has 
increased the quality of the visuals. The only remaining limitations are the amount of 
memory necessary for the storage of the images and the speed with which the huge 
amounts of image data can be processed to produce realistic moving images. 
Digitalisation of video, compression techniques and increased graphical resolution and 
colour will make the gap between the appearance and presumably the effect of both 
methods smaller. It now seems obvious that the trend towards digitisation of all kinds of 
information will also progress towards the digitisation of video. Several attempts in this 
direction, such as DVI and Apple's QuickTime, have been undertaken and may prove to 
be practical enough to become popular. 

Economical aspects 

Visualisation always increases the costs for the courseware development. A reasonable 
problem, not solved until now, is to find the cost-effectiveness point of satisfaction taking 
the important advantages and disadvantages, described above, into consideration. The 
solution of this problem is difficult not only because, one cannot estimate exactly the 
direct and the indirect costs of courseware visualisation at the beginning of the 
development process, but mainly because till now there is no reliable model to convert the 
qualitative measured effect from interactive visual-supported courseware into quantitative 
(and special monetary) figures, for comparison with the rough estimated costs. 
The economic aspect is connected with the aspects mentioned above, especially with the 
cross-cultural one through the portability problem which influences implementation. From 
the other side, the technical aspects are connected with the costs of software development 
and therefore indirectly with the level of visualisation. 

All the aspects therefore are related and so to study the main research problems a systems 
approach should be used. 

Application of visualisation in courseware engineering 

Two directions of visualisation in courseware engineering can be distinguished. The first 
and most obvious is the visualisation in courseware itself, to improve the learning process. 
The second is the usage of graphical representation, for instance in authoring tools, to 
improve the courseware engineering process. 
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Visualisation for learning 

The advantages mentioned before indicate the value of visualisation for learning. Many 
visualisation techniques are useful, both from the domain of multimedia and graphical 
methods of visualisation. The effort required to implement visualisation in courseware will 
depend largely on the development system used, but also on the availability of scanning 
hardware and on the capabilities of the development team in producing quality visuals or 
on the availability of legally usable visual materials. 

Depending on the learning content of the courseware, the visualisation can serve several 
functions. Molitor, Ballstaedt and Mandl (1989) offer a useful classification of these 
functions: 

- Representation . The representational function serves to transmit information, usually 
redundant with text. The use of the representational function of visuals depends largely 'on 
the instructional content because it has to be concrete, visible. While a verbal description 
of the face of a persons may be detailed enough to serve the educational purposes, it 
cannot be denied that a picture of a face is much easier to interpret and remember. 

- Organisation. The organisational function provides an overview or macro structure of 
the text content. This way the visualisation will probably have the advantages of the 
advance organiser, but with the extra advantage that it can easily be referenced. 
Knowledge navigation is one of the major problems in the educational use of large 
databases such as hyper documents used in discovery-type learning packages. The 
provided overview of the macro structure may also serve as a kind of navigation tool. 

- Interpretation. The interpretation function helps the reader to understand parts of the 
text, for instance by visual analogies and metaphors. An example of an interpretational use 
of visuals is the depiction of sets using Venn-diagrams or the use of histograms to show 
the development of the value of a variable over time. 

- Transformation. The transformation function enhances memorisation by providing the 
learner with extra associative links. The keyword technique (Pressley, Levin, & Delaney, 
1982) makes use of this function of visualisation. 

- Decoration. The last function, decoration, is used to beautify the text. 

The decorative function is not much appreciated by Molitor, Ballstaedt and Mandl (1989), 
because it can produce negative effects. We think the decoration function itself increases 
motivation and will therefore produce a positive effect, if not overdone. But we agree that 
an illustration serving only a decorative function will probably distract because the student 
will start to wonder about the educational value of the illustration and will spend his time 
trying to find out the deeper meaning. This categorisation of visualisation functions should 
not be confused with a categorisation of visuals, in which case one could talk about a 
decorative picture. 

Visual aids for courseware engineering 

Advocates of authoring and visual programming systems often claim that "non- 
programmers" should be able to develop their own software. However we do not consider 
programming by teachers who are not proficient in programming a good approach. The 
effort required to make good educational software is much too large to fit within the frame 
of lesson preparation. We do recognise the desire of teachers to influence the instruction 
delivered by courseware products. Adaptability of courseware will become more and more 
important over time when teachers start to integrate courseware in their normal 
curriculum. As with ordinary school-books, teachers will want to make additions after 
some years of usage. 
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Visualisation techniques may help to build systems, that enable teachers to make minor 
changes to existing courseware, such as removing a paragraph or adding one, replacing a 
picture, etc. 

Other types of courseware, like simulation, modelling and gaming require different 
characteristics from the authoring system. Modelling systems for instance are programs 
that allow the user to enter, change and calculate mathematical models. Such programs 
usually use some kind of visual representation to indicate the transfer of data from one 
function or block to the next, such as an electrical circuit where the electrical current 
passes through different components which transform the signal. 
Rapid prototyping tools have become increasingly popular in recent years. Rapid 
prototyping tools are systems that allow experienced programmers to make a mock-up 
version of software to be developed to discuss about and test the program before it's 
actually being programmed, usually in some higher programming language. The role of 
visualisation for rapid prototyping tools is obvious, because the major function of such 
tools is to produce programs that look like how the final version of the courseware under 
development should look like. It should give all the people involved in the courseware 
development process an idea of the functioning of the graphical user interface and the 
educational interaction. 
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Human Factors and muiti media design 



LexBergers 
Abstract 

^§ome human factors in relation to multimedia design will be discussed in this article. In the 
first part theory and design guidelines about human communication is applied to 'man-machine' 
communication. In the second part of the article some cognitive processes -perception and 
storage and retrieval of information- that affect multimedia design are described. 

Introduction 

Multimedia is all about communication. So, let 1 $ spend a minute on a simple definition of 
human communication: sending and receiving messages between two or more people. 
Someone is sending the message and another person is receiving it. The sender can then 
become the receiver and vice versa. 

This communication process comprises different subprocesses e.g. motivational and cognitive 
processes. 

The communication between a human being and a (multimedia) system basically follows the 
same process. There is a sender (the system), there are messages (the content), there are 
receivers (the users of the system) and there is feedback (the system reacts at the input of the 
system user). 

The major difference between 'man-machine' communication and communication between 
people is the lack of flexibility in the communication process. There are no side paths possible, 
there is always a kind of (mere or less) rigid communication. It is impossible for a system to 
act or react as a human being, to understand what troubles the user has or why the user is not 
able to proceed. 

However, designers of multimedia systems can take into account Iranian factors' as much as 
possible to let the communication process be as optimal as possible. By liuman factors' I mean 
the psychological attributes that are involved in communication processes, including 
motivational processes and cognitive processes like perception, attention, memory, learning 
and thinking. 

In this article I will concentrate on some important notions about communication and on two 
cognitive attributes: perception and memorisation (storage and retrieval of information). 

Although psychology is a very young science, designers of multimedia systems can benefit 
from it. To become a good designer, especially a user interface designer, it is most needed to 
have at least some knowledge about human factors. 

So, the discussion is about what human factors can bring in order to develop guidelines for 
multi media design. 



31 

33 



Sending messages in communication 



You probably know what coins are used in the United States of America. So you should be 
able to solve the next problem. T have two coins that total 30 cents. One of them is not a five- 
cent piece. What are these coins? 

Do not read further now. Think first about this problem before you read the solution. 

Of course, there is a triclc The solution is a five-cent piece and a quarter. Why? One of the 

two coins {the quarter!) is not a five-cent piece! 

Most people cannot easily solve this problem, because they trust communication conventions 
too much, like the one that people trust that when there is some special condition, it will be 
described. If special conditions are not mentioned, then the 'receiver 1 can assume there are 
none. 

In designing multimedia systems this means that the system has to always inform the users 
about special features that are not visible at first glance. The user interface must reveal the 
special features. The user should be able to perceive the message in its context. If the context 
is absent or unclear then understanding becomes more difficult. 

Another communication convention is that people normally do not mention obvious, 
unimportant, or unlikely things. When people do mention them, it conveys a special signal 
about their importance. 

Here is a danger, because the sender (the designer) determines what will be obvious to the 
user and what will not. The only way to avoid problems is to do several user tests and find out 
if it really is obvious to the users. 

Communication is not always very effective because of inconsistencies and ambiguities in the 
process. This does not mean it is not possible to have effective communication. What is the 
best way to send messages to a computer user with whom you are trying to clarify your 
intentions? There are some rules to be aware of: 

Consider the knowledge of the user 

Always consider the knowledge and understanding of the system users. The users often do not 
have the same background you as a designer have, they may have an entirely different concept 
of what you are speaking about. Do not use terms the system users do not understand. Try to 
avoid difficult and ambiguous terms, if this is not possible define them. As much as possible, 
let the system speak the language of the users. 

Of course users differ in capability and experience, try to design with these differences in 
mind. 

Present complete messages 

Ensure that the messages in the system are as complete as possible. Give explanations where 
needed. 

Do not use negatives 

Negatives in a sentence will be read slower and less accurately than comparable sentences 
phrased in affirmative terms. 
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Provide messages and orientation possibilities 

Provide messages informing users which part of the system they are in. In large systems users 
may forget which facility they are using, beacuse they have navigated through many functions. 
A support function showing them where they came form and where one can go to, might be 
very helpful. 

Save user's effort 

Try to save user's effort when operating the interface. This can be realised by using the 
minimal number of dialogue steps necessary. For example, do not use two question and 
answer steps where one will be sufficient. 

Reduce the amount of typing 

Reduce the amount of typing and the use of abbreviations and codes. Long winded dialogues 
may be supportive at first but users quickly learn dialogue steps. Slow and multi-step 
dialogues soon become frustrating. 

Use defaults. 

Set default replies where there is a predictable answer, this saves the user work. 
Help 

Provide on-line help whenever possible. Help has two functions, first as a learning aid for 
users who are reluctant to read manuals, second, as a memory support for experienced users 
who need confirmation of some detailed aspect of an operation. 

Undo 

Mistakes will be made and users will want to backtrack in a dialogue the sequence they have 
gone through and start again. The interface should provide the ability to go back (or go 'up' in 
a hierarchical system) and recover a previous state. 

Consistency 

The format and the execution of commands should be consistent throughout the interface. 
Consistency reduces the amount users have to learn about an interface. 

Presenting feedback in communication 

In multimedia systems, feedback is the user's key source of how to complete a task. Feedback 
takes away the fear from the user, it helps the user to feel at ease because it confirms that the 
system did understand him (and visa versa). Feedback makes it possible for the user to 
correct words or actions and it gives the user new incentives. 

If feedback messages are not well expressed, it will disturb the communication process. If you 
give feedback that is not useful at that moment or if the feedback does not tell what to do 
instead, it does not help at all. For example: 

If one receives the feedback message "You have to select another function," you do not know 
what to do, or which function you should select. Feedback has to be constructive, it must help 
the user to proceed. A good example is: "You have to select the delete function." 

If a user receives negative feedback, he will not feel at ease and this can be demotivating. It 
will be more motivating for the user to receive positive, constructive feedback. For example: 
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Instead of stating that a chosen option is not available, state that the chosen option is available 
after certain required work. This will help the user establish a positive attitude towards the 
system. 

Effective feedback needs to be presented at the proper pace and should be tailored to the 
specific user situation. 

Cognitive Processes 

The world around us communicates to us by sound, colors, shapes, signs, symbols, etc. This 
ongoing communication between the Svorld' and humans facilitates cognitive processes like 
perception and the storage of information in our memoryand the retrieval of it. 
In the next sections will be discussed this particular cognitive processes with relation to the 
design of multimedia systems. 

Perception and pattern recognition 

How many times have you recognised telephone callers from their very first words? We can 
often recognise a musical piece from the first few notes. Sometimes we can identify a friend 
or relative from a cough or a footstep. And from one quick look at a computer screen you 
know someone is working with Windows. In other words: people are very good in 
recognising patterns. 

People are capable of recognising familiar patterns almost instantaneously. At least two things 
about the examples above are striking. 

First, recognition occurs almost instantly. Second, we seldom make mistakes. 

Patterns do not have to be analysed to be interpreted. We do not analyse a face in detail but 

recognise it at once. The face is perceived as a whole and instantly interpreted. 

When designing a multimedia system you will benefit from using the principles of pattern 

recognition. Designers already benefit from the fact that humans are good pattern recognition 

Machines'. The use of icons is an example (the pictogram will be understood as a pattern 

computer users recognise immediately) , and also the use of multiple windows (each window 

represents a pattern). 

In many more design situations you can make meaningful patterns. For example: 

Organize data to be shown in easily perceivable patterns. 

For example, use a bar-graph to present data instead of a pie-graph. The bar graph 
gives a better 'pattern' , because the differences between the data will be perceived at 

once, while a 

pie graph needs more mental activity to interpret. 

It will also help people to perceive information when you present lists in an orderly 
way. Make distinctions between the columns in a list by using white spaces. Each 
column will be perceived as a pattern and will be recognised immediately. 
When you have to present long text on the screen, divide the text into paragraphs 
using white space between them. Avoid too much text on one screen page, because 
this will hinder the recognition of a pattern. 

And when you design several windows, each with different functions, make sure that 
the patterns of information and fields are different between these windows. This helps 
the user recognise immediately the right function in these windows. 
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Finally, be consistent in presenting information. The more consistent something is, the easier 
it is to see patterns within it and hence its structure and characteristics. 



Mistakes in perception 

Most interaction with a computer system starts with visual perception: the computer user 
locates a menu selection, an entity to be deleted, the cursor or he recognises a form or shape. 
Unfortunately, people can be wrong in interpreting what they see. Most famous in everyday 
life, are the witness statements of how a traffic accident happened. Often, different witnesses 
have their own story about the accident, confusing the police. The explanation of this is that 
people tend to see what they would like to see or expect to see. 
What we can learn from this is that people make mistakes in perceiving an user interface. 
Maybe they do not notice status-information, or they interpret messages, pictures, or 
processes wrongly. They will have problems in reading codes and unfamiliar abbreviations. 
So for users to perceive the system well, information must be: 
As familiar to the user as possible, 

Legible: the text does not flicker and is easy to read at a glance, 

Distinguishable: the figure is clearly separated from the background, 

Comprehensible, 

Uncluttered, and 

Meaningfully structured. 
Furthermore, to minimise the use of unfamiliar codes and abbreviations, one should make 
schematics and pictures as clear as possible to allow the user to recognise and understand 
them at once. 



Perception and colour 

Colour Perception 

The challenge for the artist is to capture colour in a form that creates mood and atmosphere. 
For the interface designer the challenge is to use colours as a form of coding to display 
information in the most efficient way. Unfortunately, many designers new to colour graphics 
have tried to become artists. The result has often been disastrous: a mismatchof colours which 
constitutes neither a masterpiece nor an effective interface. 

There has been a tendency to use as many of the available colours as possible in user 
interfaces. Moreover, there has been a preference to use highly saturated colours based on the 
underlying assumption that the brighter the better 1 . But if you ever looked at some of the 
original videotex pages on television you will know how garish such a display can be. 
Sometimes it is almost impossible to extract the information required. 

Colour Junctions 

Colour is most powerful for search tasks and of little use in tasks requiring identification, 
categorisation and memorisation of objects. In comparison with achromatic coding (black and 
white), colour has been shown to be of more use in search tasks for inexperienced users than 
for the experienced. 

Colour is a very powerful way to divide a display into separate regions. Hence segmentation 
of a display for detection and search tasks is very useful. Areas that need to be seen as 
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belonging together should have the same background colour. Too many colours will increase 
search times, so colours should always be used conservatively. 

There are different psycho-physiological theories about the use of colour. 

Below is a summary of some key points that will be helpful in designing user interfaces: 

Looking at colour screens is less tiring for the eyes than looking at screens with no 
colours. If it is possible to use colours, use them! 

It takes fewer eye-scans to locate objects correctly coded in colour than those coded 
by other methods. Use colour to mark objects! 

For most applications four to seven colours is the maximum suggested number on a 
screen at one time. The amount of colours depends on the application. A multitude of 
colours may be valuable in a virtual reality design, where as in text the use of too many 
colours can result in an overwhelming and potentially confusing presentation. We 
advise you to use no more than three colours for text. 
No clear evidence exists to suggest what colour should be used in presenting 
text. There are a number of claims that yellow (on a dark background) is the best 
colour for display characters because the eyes are most sensitive to its wave length. 
This seems true when the eyes are adapted to a typically lighted environment. When 
people are working in a dark environment the eyes are more sensitive to colours at the 
lower end of the visible spectrum (blue-green range) 

To make the best choice you have to find out in what type of environment people are 
working with the Multimedia system! Other studies about the use of colours on a dark 
background indicate that reading performance is better on yellow, magenta, cyan and 
red, rather than green and blue. 
In general, the best way to use colour is: 
Background: 

cool, neutral, pale colours 

Cool colours (blue and green) make objects look smaller. 
Foreground: 

warm, bright, strong colours 

Warm colours (red and yellow) make objects look 'bigger'. 
Many colours have some symbolic value in society. Red is used when danger occurs, 
or for something that is not accessible. Green symbolises nature and environmental 
issues, but also safety or a go ahead condition. Be aware of different meanings of 
colour in other cultures. For example, Eskimos have more perceptions of (and names 
for) 'white 1 than western-European countries. 

Computer users can link items more easily when they have the same colour. This 
means that colour related items can not be used for other purposes at the same time. 
About 8% of the male population are colour blind, they are unable to distinguish 
between various colours, especially green and red. This is important to know when 
designing a system. It also means that you have to complete some tests involving 
colour blind people! 
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Perception and 3 dimensional aspects 



The world appears to us in three dimensions. This allows us to see depth and perspectives. 
Since people are used to the perception of reality, one of the most effective user interfaces 
design techniques is simulating 3-D interfaces. 

Three dimensional interfaces enhance the perception of affordances. The affordances of an 
object refers to its possible functions. A chair affords support whether for standing, sitting, or 
the placement of objects. A pencil affords lifting, grasping, turning, poking and of course 
writing. 

In design, the critical value is perceived affordances: what people perceive the object can do 
(in multimedia systems: icon, button, menu bar, etc.). We tend to use objects in ways 
suggested by the most salient perceived affordances, not in ways that are difficult to discover 
(Hence the fact that many owners of electronic devices like videorecorders often fail to use 
some of their most powerful features. They often do not even know of their existence). 
You are probably aware of the next examples: 

Buttons are lumps to push on 

Data entry fields are holes to put things in. 

Graphical User Interfaces (GUI), using (overlapping) windows (like pop ups), simulate this 3- 
D perception. If there's not a GUI-environment available in your project, or if you use only 
presentation material (slides, Powerpoint application), you can benefit from the following 
perception insights about 3-D. 

Size 

If two examples of an object are considered to be roughly the same size then the smaller one 
appearing on a 2-D surface is perceived to be further away than the larger one. 

Interposition 

If one object partially obscures a second object then the blocked object is perceived to be 
behind and beyond the blocking object. 

Shadow 

Shadows cast by an object provide some cues to the relative position of objects. 
Texture 

As the apparent distance increases, the texture of a detailed smface becomes less grainy. 
Motion 

Objects that are further away appear to move more slowly than objects that are closer. This 
effect is one of the most important cues that enables us to perceive distances and depth. 

Memory and multimedia design 

Memory is one of the critical limiting factors of human information processing which affects 
the design of multimedia systems in many ways. I will discuss two aspects of memory : the 
storage of information and the retrieval of information. 
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Storage 

The concept of human memory consisting of a short term and a long term memory is of course 
well known, since computers also have a RAM memory (short term) and a hard disc (long 
term). 

From the theory about short term memory we know that it has a limited capacity. 
Psychologists found in experiments that the limit is 7 items plus or minus 2. Items are not 
'bytes' but 'chunks' of information. Chunks can vary from simple characters and figures to 
complex abstract coi.cepts or images. 

This means that in a multimedia system the amount of information that has to be remembered 
by the user has to be minimised (minimum of the number of actions, commands, objects, 
properties, rules, etc.). If users have to remember information for some time, make sure the 
total amount of items to be held in memory does not exceed Jive. 

Organisation 

Because our long term memory (LTM) is structured in a certain way, we are able to retrieve 
facts, procedures, principles, experiences, etc. relatively fast. This organisation is certainly not 
structured like the hard disk of a computer. Humans are able to make flexible connections 
across 'directories', 'sub-directories' and 'files'. The associative power of the brain exceeds far 
the power of computers. 

Our long term memory is organised in several overlapping ways. The basic organisation of 
LTM is thought to be semantic. Data are stored in terms of linguistically based concepts 
linked together in a highly developed network of meaningful categories. For example, if you 
are mentally checking your vacation outfit and start with your passport, you will continue with 
other paper work like your driving licence, insurance papers, money, address book, flight 
tickets, etc. In some way you have made a category of all the paper work. Fortunately each 
piece of paperwork is related to other categories. For example, the flight tickets are also 
related to: the airport, the airline, the pilot, tax free shopping, the flight reservation, etc. This 
means that our memory is not organised into discrete, non overlapping categories, but is 
organised in a flexible way. 

We can learn from this when we design multimedia systems. In your multimedia design you 
have to categorise as many tasks, items, options, and functions as possible in a non- 
overlapping way. If you design a Graphical User Interface the use of windows directs you 
more or less automatically in categorisation of tasks, items, information, etc. You should not 
only categorise menu bar items, but also lists, icons, buttons, data entry fields, etc. You should 
make it clear to the computer user what relations exist between those categories, allowing 
them to retrieve information in a flexible way. Also, with your design, encourage users to 
categorise information which must be retrieved later. Provide prompts, labels or (menu) names 
which are relevant to those categories. 

Memory is also organised in episodic memory. In episodic memory we relate a word, an 
experience, an 'odour*, etc. to some ,vent. For example, when you try to retrieve the name of 
the author of a novel, you recall one evening when you spoke about a book by this author. In 
your mind you see the table with a glass of wine and the book next to it. You remember the 
atmosphere of that evening and other features, that brings you to the name of the author. 
It takes less effort to remember items from this episodic memory. This explains why 
metaphors are used in graphical user interfaces, because they relate activities to human 
experiences and events. 
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Retrieval of information 

Psychological research has revealed that the retrieval of information from our memory is far 
better when we can recognise stored material rather than recalling material from memory. 
This is why the 'multiple choice 1 questions in tests are so popular. It is also one of the reasons 
people like multimedia, including running video, pictures and windowing techniques. 
Our ability to recognise information is particularly remarkable for pictorial information. 
Several studies have shown that our capacity for recognising pictures may be almost unlimited 
and is extremely accurate. 

Psychologists found that wrong interpretations of commands occurred far more often when 
using command names (text) than icons (pictures). The obvious implications for interface 
design is that you have to use recognisable, pictorial items instead of textual items that a user 
lias to recall from memory. 

Computer systems are very valuable in helping people retrieve information. In fact the memory 
burden of computer users can be drastically decreased (especially, as we have seen, when they 
are using systems with a Graphical User Interface), since they only have to recognise icons, 
menu options, and pictorial information. 

Graphical User Interfaces however do not completely decrease the memory burden. People 
will always have problems retrieving information, even when they are using a supporting 
computer system. This is related to: 

Distraction 

When people are distracted during their task performance, they will forget recently learned or 
stored material. Even a small number of simple 'chunks' of information are lost within 20 
seconds if there is distraction during input. 

In your interface design you can help people if they are distracted. Users must be able to stop 

an ongoing process temporarily and users must be informed about the status of the task 

(status information that answers user questions like Where am I?). 

If the distraction means that the user has to leave the current task or module, and has to 

switch temporarily to another task, the user must always be able to go back where he came 

from. 

Interference 

Other inputs which impair retrieval are called interference. When you are trying putting some 
material in your short term memory (for example: some names of clients), and there is at that 
moment other input you have to attend to, you will have trouble retrieving the former 
material. 

Supplying irrelevant material during input to short term memory makes retrieval more 
difficult. 

This means you have to present one task at the time in the Multimedia system. If you can 
avoid it, do not confront computer users with other inputs at the same time! 

Primacy and Recency Effect 

Psychological experiments show that when people have to remember information, the last 
items of the received information are best remembered (recency effect). 
The middle items are poorly recalled, and are even not recalled when there is a lot of 
information. The remembering of the first items, however, is also very good (primacy effect). 
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This means that in presenting information to the computer user (e.g. text) the most important 
information has to be presented at th? end (or at the beginning). For example present 
explanations of how something is working in the middle, but present the concrete "how to do' 
things at the end. 



40 

42 



Conceptual Metaphors for the Visualisation in Interactive Learning 

Piet Kommers 



Abstract 



With the arrival of the full spectrum of presentation modes in computer-based learning 
resources, it becomes urgent to review already existing theories about visualisation and 
animation as defined for paper-based publication, movie and video. 
One of the differences between traditional media and hypermedia is that the first ones 
take the user along in a consistent sequence of episodes that refers to a lay-out, story 
or scenario that can be understood and in some way or another "be predicted' from 
cultural experience and prior knowledge. Hypermedia in contrary allows the user to 
jump away from a current piece of information to a certain aspect which is only an 
arbitrary detail of the previous scope. Text-based hypermedia (called hypertext) allows 
the user to browse via concept identity. This means that a term in an explanation can 
be exploded by clicking the location of this tenm (called hot spot). The same term will 
appear at the top of the exploded view, and will also be the key issue its content. The 
concept identity is the fact that the term in the clicked hot spot is identical to the 
subject of the exposition in the exploded view. Browsing through pictures, video and 
sound elements allows the user to zoom in on details. However it is quite complicated 
to guarantee some kind of concept identity in the transition from one picture element 
to another. The reason is that picture elements are less self-contained than words in a 
text. The context dependency of visual elements causes surprise and confusion in the 
user. One way to solve this problem is to enclose meta-information with the visual 
representation which links different views with concepts that are present at both sides 
of the link. 

Hypertext links can be visualised in a concept network, where the nodes stand for 
concepts. The concept node represents the : dentity between hot spot and the key issue 
in the subsequent exploded view. Generalising the concept network to picture 
transitions can be done by prompting clusters of elements and labelling them with a 
concept term. The choice of clusters and the term will orient the user to the type of 
digression he/she may expect after clicking a certain hot spot. 

Conceptual overlays in audio and video segments presuppose a moderate level of 
granularity; The overlay should remain valid for a minimum time span so that the user 
can anticipate his browsing direction while seeing the next fragment. This can be done 
by having the hot spot elements at static positions in the screen, while the moving 
elements settle the context situation in which critical concepts arise. The conceptual 
metaphor for effective use of hypermedia constitutes schematic representations which 
is an accentuation of the meaningful elements the user should be aware of while 
browsing through unknown information. At the same time it might function as an 
epistemic analysis for the hypermedia designer who needs an explicit content 
description in the design phase. 
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Introduction 

Computer-based learning programs have triggered again the basic discussion on 
didactic strategies, styles in learning dialogues and the design of information elements 
like text , visual images and sound. The key element in this revived debate is the notion 
of 'self-control' for the student versus the attempts to facilitate learning by regulating 
the learning process by pacing and sequencing information elements and systematic 
remediation of evolved misconceptions in the student's mind. Hypermedia and multi- 
media were quite welcome after twenty years of high expectations of 'advanced 
programmed instruction', 'intelligent tutoring' and 'student model-driven dialogues' 
Rather than stimulating the traditional opposition between student- versus system 
control its seems more productive now to unify them by asserting that hypermedia 
explorations might serve complementary stages in learning and instruction. Initial 
stages of learning in which pre-cognitive levels of expertise and intuition in the student 
are mobilized, versus the need for a more controlled instruction facilitating a later, 
more operational stage of learning in which correct task performance plays an 
important role. This chapter presents the merits of recent visualisation techniques in 
pre cognitive learning stages while using hypermedia. Essential for the cognitive 
benefits in this stage are the so-called metacognitive effects; Balajthy, E., 1990 and 
Cates, W., 1992. 

The availability of pictures, sounds and video in electronic learning material brings the 
opportunity to confront the student with flexible realistic views in a certain domain. 
Especially as competing media programs like television commercials with astonishing 
video clips, animations and highly suggestive and condensed social episodes penetrate, 
it is quite urgent to make interactive learning programs as expressive as possible to 
elicit emotions and keep the attention of the user. 

The user flexibility of hypermedia gives full control on sequence, focus and perspective 
to the learner. While hypertext browsing has become popular quite quickly, 
hypermedia browsing is still rare. One reason for it is the amount of effort to prepare 
the numerous transitions and necessary visual consequences that should be anticipated. 
Another important complexity is in the inconsistencies that may arise when allowing 
the user to browse between visuals, sounds and video fragments. The chosen solution 
to overcome ambiguous transitions between pictorial elements is to superpone 
conceptual structures on the candidate hot spots. Before going into the techniques of 
concept representation, - metaphors and interaction procedures, let's review already 
existing theories about visualisation and animation as defined for paper-based 
publication, movie and video. 



Words, Images and the Level of Abstraction. 

Words and images cannot be derived from each other, ' Foucault, 1966. It stresses 

the uniqueness of human senses. Quite typical prehistoric tribes reflected their desires 
and fears in pictographic images. After centuries however they moved to more abstract 
cryptic symbols representing objects but also feelings and reflections These symbols 
did not necessarily resemble the objects they referred to: A separation between reality 
and mental representation Words like symbols do not necessarily refer to one concrete 
object or situation, it may r'so point to a more general unity of things Words like 
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'cattle' or 'agriculture' are typical generalised concepts in the sense that they 'pack' a set 
of more specific concepts like 'cow', 'deer' etc. 




The typical difference between word and image is that words do typically tend to 
express generality and tend to neglect individual perceptions, while images do defacto 
point to specific elements of reality. The trade-off for hypermedia designers is to take 
the advantage of words, tending to tell the most general impact of their meaning, or 
images which se -;e the user to the uniqueness of a certain element. General concepts 
like 'inflation', responsibility', 'entropy' or 'inertia' are difficult to express in static 
images. If we try to do so as they can hardly express conditions like 'sometimes', 'only 
if etc. and it brings the user in the danger of picking up irrelevant or even uncorrect 
attributes in the visualisation of the key concepts. At the other hand images can sketch 
a typical situation quite rapidly by triggering perceptive templates that we as users 
know from past experiences Above the so-called 'scheme-activation', images, sound 
and video have a strong appeal to the imagination of the perceiver. Spatial and episodic 
consequences of the scenery can be generated and remembered for a long time. 
Conflicts in pictures can be detected quite easily, as the human eye has been trained for 
many years. One could even say that the human brain is especially equipped to interpret 
reality from sociaJ and visual messages. The picture with the bed scene in figure 3 is 
ambiguous in many senses, but several hypotheses can be rejected by the observer 
immediately The perceptive and interpretative effects are quite dependent from 
conflict elements in the image. Archetypical features in the outlook and character of 
actors make it easier to arrange a conflict in the scene. Conflicts arise emotions in the 
spectator and makes it easier to catch the attention for a longer time. Learning effects 
will only take place if the arranged conflict is anticipated by a question or open interest 
in the user. The first of the two next pictures will hence evoke a longer attention from 
the viewer and will also be processed at a deeper level. As a consequence it may be 
expected that the picture in figure 4 will be remembered for a longer time, compared 
with the pictaie in figure 5 which causes less arousal at the spectator. 
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As one tries to explain the difference in examining figures 4 and 5 it makes sense to 
distinguish between 'understanding' (coping with the meaning of a certain impression) 
and 'remembering'. It is generally assumed that meaningful processing facilitates 
remembering. Semantic processing for instance is assumed to be more effective than 
formal processing; (Jenkins, 1974). Increasing the 'depth' of processing will produce 
better memory (Craik and Lockhart, 1972; Craik and Tulving, 1975). The key issue is 
to arrange the images so that they actually trigger emotions, imagination and personal 
experiences from the past. As culture changes quickly, and persons differ quite a lot, it 
is hard to find overall guidelines. The urgence to chose the right elements in visuals is 
clearly demonstrated in the effort given to commercials. 

Trade off Between Abstraction and Concreteness in Presentation. 

So far the global distinction betveen visuals and audio (perceptual) at one side and 
verbal or written information (conceptual) at the other side has been made. While 
hypermedia designers are steadily improving the flexibility and expressiveness of 
browsing, the more fundamental question arises to what extend we might expect 
beneficial cognitive effects for the user. A dominant criterion so far is the level of 
excitement and entertainment: The more exuberant presentations, inclusively three- 
dimensional impressions and direct manipulation in space, the more is the appeal to the 
user. 

Besides the alertness on cost-effectiveness it becomes important to find a rationale for 
the learning criterion that might decide upon presentation richness. Edgar Dale took up 
the concept of 'realism' of Hoban and Zissman (1937), and laid it down in his 'Cone of 




Experience' Dale's ambition is to adapt the realism and the level of user interaction 
during media presentations to the momentary user's level of cognitive development and 
competence. The trade off has to be made between the conciseness of information, 
which is typically appropriate if the student can read and understand scientific 
expressions, mathematical formulas and pictographic symbols at one side, and more 
elaborate experiential situations which cost more time and instructional effort, but 
which will compensate lacks of prior knowledge and which will allow the student to 
learn by experience before any formal instruction takes place. 
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Figure 6 



Dale's notion on levels of presentation goes beyond the question how to visualise. In 
fact it addresses the issue of interaction style and instructional strategy. For 
hypermedia design we can use Dale's cone to decide upon when to invest in full motion 
video, still video, sketches, the combinations with audio, or visual and verbal symbols 
only. As many student characteristics are hard to assess during or even more difficult 
before a learning session, hypermedia presentations might be superior to prepacked 
CBL sessions as they leave essential choices about the modality of information to the 
user. Based on cognitive style, prior knowledge or momentary interest the student may 
ask for presentations at a lower (more concrete) level. It may even be the case that the 
student at a certain moment wishes to interrupt the presentation and asks for a 
confrontation with a simulation environment in wliich so-called 'what-if experiments 
can be done 
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Concepts, Objects and Visual Representation. 



The idea that one can think in images is not as strange it sounds Of course its validity 
is quite dependent upon our definition of 'thinking'. Many of the human actions are 
directly based on what we see, hear and smell; The resulting behaviour at those 
mentioned moments is controlled by a very fast cycle between perceptions, emotions 
and testing our feelings by taking a perspective for new impressions again. 
Abstractions, reasoning and conceptual change occurs hardly in those situations. This 
is the very reason why certain social behaviour and emotional reactions are hard to 
change. It is this pre-cognitive aspect that plays an important role in the effects of 
visualisations. At one hand we should be aware that images and sounds trigger a 
fundamental layer in the user/learner, while at the other hand we know from research 
that they are most effective to consolidate the process of knowledge acquisition and 
conceptual change. The user of a hypermedia system may benefit from a flexible access 
to different visual representations so that we might describe this browsing behaviour as 
operating a camera: Panoramic view, zooming in, zooming out, filtering, jumping back 
etc. If the information nodes and links via hotspots have been defined from a didactic 
perspective however, we might expect that these perceptual transitions finally promote 
mental views which are more or less congruent those of the experts who prepared 
them. The central discussion about hypermedia as learning resources is to what extent 
explicit conceptual arguments should be added to the browsing space, before effective 
learning takes place. This discussion is both relevant for optimising hypermedia for the 
pre-cognitive stage of learning, and for the integration of hypermedia in more 
controlled instructional episodes as well. 

Let's imagine the domain of combustive engines and see how visuals may facilitate 
cognitive stages in understanding, generalisation and problem solving. It's hard to 
imagine how difficult it would be to teach the typical characteristics of a diesel motor 
to someone who never saw, smelled and felt the movements of a running diesel engine. 
Not knowing the physical and mechanical principles of a combustive engine, one will 
still derive quite important aspects of the combustive mechanism by seeing, hearing the 
effects of such an engine built in a car, boat or train. Having observed the starting 
process of combustive engines gives already the notion that certain conditions must be 
met before the process starts running. E.g. the effect of temperature, the speed of the 
rotation and the degree of choking and giving throttle. 
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Figure 7 



However most of the even very successful car drivers do not know any formal aspect 
of ths combustive mechanism. Even after many years driving, these perceptions and 
experiences are not sufficient at all to build up a correct idea what happens in the 
motor itself. To acquire such knowledge is exactly the typical task for education and 
training. In terms of Dale's cone of experience it is the question then to migrate 
tactically from perceptual confrontations with the phenomenon of combustive engine 
up to more conceptual representations so that flexible knowledge may evolve. 




Otto Engine Figure 8 Otto Engine Figure 9 
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The three images and caption text of the 
combustion engine are clipped from 
Microsoft's Encarta Encyclopaedia on CD- 
ROM. 

Early Internal-Combustion Engine 
One of the most important inventions of the 
mid- to late- 1800s, the internal-combustion 
engine generated mechanical energy by 
burning fuel in a combustion chamber. The 
introduction of the new engine led almost 
immediately to the development of the 
automobile, which had been largely 
unfeasible with the unwieldy steam engine. 
Shown here is a 1925 Morris engine, the 
basic unit for a family car. It features four 
in-line cylinders with aluminium pistons. The valves are opened by push rods 
operated by a camshaft and closed by springs. Power is transmitted by means of the 
crankshaft to the gearbox. (By: Dorling Kindersley) Otto Engine Figure 10 




The displayed opened engine takes place in an animation sequence to illustrate different 
periods in the Otto-cycle engine, named after its inventor, the German technician 
Nikolaus August Otto (1832-91). It is the familiar gasoline engine used in automobiles 
and aeroplanes. The diesel engine, named after the French-born German engineer 
Rudolf Christian Karl Diesel, operates on a different principle and usually uses oil as a 
fuel. 
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The first image (Otto Engine 
Figure 1) gives quite a global 
impression of the overall shape 
and is quite close to a 
pictographic representation; The 
global components are visible, 
only to attend global positions of 
the crankshaft and two of the 
pistons. To illustrate the 
interrelated positions of the 
crankshaft, piston rod, piston and 
valves, the front view is more 
appropriate as it gives a natural 
argument to display only the 
constellation in one of the four 
cylinders. 

Figures 8 to 10 displaying the 
Otto Engine with an increasing 
level of detail illustrate the 

capability of current drawing facilities in graphical user interfaces to prune away 
successive levels of details, even in bit-mapped images. The potential opportunity of 
these various smudging levels in hypermedia presentation is that we might confront 




Figure 1 1 



Front view 




students with pictures that are automatically filtered from details just above to the level 
of understanding at that moment. 



Computer-based Adaptation of Visual Images 

As more tools for image processing become available, it becomes easier and 
challenging for courseware engineers to prescribe adaptive levels of detail in didactic 
images. The next two pictures of arch fences start quite differenct reactions in 
perception, interpretation, generalisation and retention for the user. Figure 13 has been 
derived from figure 12 by means of Corel Draw's 'tracing 1 facility. 




Figure 12 Figure 13 

To illustrate the technique as used to change Otto Engine Figure 10 into 9 and finally 
into 8, the bitmap image of the thr r ?e arches above is used to trace its contours and 
finally drawing the attention of the user to i k .s major components, hiding the structure 
of the bricks the fall of the light etc. Essential however is that the firal trace image 



keeps the impression of a real object, with small vividness in its lines so that it activates 
the imagination of the spectator. Visuals, composed of purely geometric elements miss 
this suggestion and though they suggest more exactness, they lack the appeal to the 
human mind. Finally the contour image in figure 13 costs only 200 times less storage 
space and can be drawn in a fraction of the time compaired to figure 12 Ir *act this 
contour sketch can be handled as an object image, which is easy to colour or to rescale 
based on an actual presentation or instructional arguments. In terms of Dale's 
dimension of contrived versus real experience it is obvious that the contour image is 
situated more closely to the symbolic level, however it keeps a stronger reminiscence 
to the perceptual image of the photograph by its suggestive details. 

Bit mapped images are difficult to manipulate in a meaningful way, as there is no 
information about its components. Object images can be filtered, cropped and distorted 
based on semantic criteria as long as the names o;its labelled parts are compatible with 
the descriptions of its functions and relations to the presentation arguments like speed 
impression, articulation of perspective etc. 




Figure 15 
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Figure 16 

Object oriented visualisations permit the hypermedia designer to be more expressive in 

the functionality of certain objects and to 
change easily between certain cognitive 
perspectives elicited by display arguments 
as shown before. The display of the 
pneumatic train semaphore (figure 17) 
gives a rudimentary impression of its 
working. The human eye however needs a 
longer time to detect the mechanistic 
elements like the airtubes, the cylinder, the 
piston and the final drive mechanism of the 
switch staves. 




The help of arrows to indicate the 
direction of movements and the cut and 
opened parts ask quite an effort of the 
visual designer to reduce the complexity 
while keeping the picture appropriate for 
different levels of expertise in users. 
Changing the function of the display in the 
explanation chain needs a laborious 
redrawing, and of course it cannot be 
generated automatically by a computer 
program. Is easy to see that one of the 
major options in using object-oriented pictures is in the pick-and-mix facilities, 
enabling hypermedia designers to include precisely defined elements of other pictures 
in new ones 
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Let's focus on technical drawings like those to explain maintenance personnel where to 
find certain elements or to prompt them in which order procedures should be 
performed. These drawings need sufficient details in certain parts while they may be 
global in the peripheral zone. Good examples can be found in repair manuals for the 
car dealers. 




Within the scope of a certain assembly action like tightening the bolts through the 
cylinder head (figure 18), a drawing is made so that it servers as a direct orientation for 
the operation to be performed. The location and relative positions of the bolts have 
been drawn quite precisely, while the surface of the motor head is only suggested 
vague. A more delicate action to be expressed is the adjustment of parts, as show u in 
figures 19 and 20 Especially in the last two examples it it obvious that one has to ftnd 
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a balance between the clearness and convenience of overview at one side and the wish 
to combine different actions in one picture at the other side 



Concept exploration by visual negotiations 

Visual images may play an important role in concept formation, especially if the 
application domain is visual. If the concept space is more abstract, concept mapping 
techniques can be useful to express global entities as a framework for elaborations later 
on. Concept mapping has become popular in learning, design and problem solving The 
underlying idea is that the schematic representations give the user a greater flexibility 
while searching for creative solutions. Graphic representations rather than oral and 
written words enable to express the simultaneity of ideas in one stroke. Concept maps 
are mostly drawn in twodimensional space. For more dense networks, a third 
dimension may help, especially to distinguish between fore- and background. The 
negotiation metaphor is well known for learning purposes and says that students 
should not simply be receptive for new information, but they should be placed in the 
position of a stakeholder who faces the trade-off between changing him/her self (by 
internalising the new concepts) or by analysing the new ideas for their structure and 
reformulate them so that they fit in their actual state of knowledge. The first approach 
asks for a better memory while the second heavily relies on reasoning and detecting 
analogies. The concept mapping technique as a negotiation tool for evaluating these 
two strategies can be intensified by introducing 'spatial impossibilities' like present in 
the well-known paintings of Esscher. The two images below place the spectator in a 
mental conflict. He/she has to decide upon fore- / background positions based on the 
content in the nodes (placed on the edges of the figure). 





Telematics 



Care 



Figure 21 
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The choice should be effectuated in dragging the concept terms so that he spatial 
connotation looses ambiguity and reflects the semantic implication as well. The 
cognitve benefits of provoked mental conflicts will only emerge if the student is 
trained and motivated for a certain level of cognitive flexibility. If we regard the typical 
atmosphere and discipline in educational and training setting we see the opposite 
tendency: Students are trained to assimilate prestructured views and opinions. 
However many students face massive uncertainty if they meet realistic problems they 
have to solve, because they don't have fitting solutions ready to apply in their memory. 
So for creative jobs like design, human managment, boundary spanner and trouble 
shooter it is obvious that cognitve flexibility is a defacto attitute, that should have been 
stimulated in the early phases of learning. 




Figure 22b 

Spatial connotations of concept relations have been worked out by researchers quite 
intensively so far The basic assumption is that human knowledge cannot only be 
scaffolded by verbal concepts, but need a more perceptual and experiential 
representation for several reasons Computers may help in realising the manipulation 
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and reconciliation between the user and his/her conflicting mental perspectives. In fact 
the visualisation here works as a provocation to the user who will strive for equilibrity 
between knowing, feeling and seeing. The perceptual conflict as demonstrated above 
can also be extrapolated to an operational one, in which the user has to compete 
against 'primitive' intuitions how to achieve a certain spatial effect Let's take a s!nple 
spatial, conflictless design tool for concentric objects before going into an example 
which may illustrate provocation^ situations for travellers through concept space. 




Figure 23 



Figure 24 



To show how we as perceivers are bound 
to 'common views' try to identify the next 
image. Where can you find this object? If 
you have problems finding the answer, try 
to make assumptions about the genetic 
element of this object: How did this shape 
evolve from the centre part? Why is the 
density of smaller elements higher at the 
outside of the object? 
If you still have problems, please study 
some of its substructure in the next 
picture 
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Probably the organic structure in the centre of the figure acknowledges your initial idea 
that it has to do with a tree. However the trunk fails. If I ask you: "O.K. it is a tree, but 
where can you find such a strange shape of a tree? 1 ' You might start some reasoning 
and find out it must be the top view of a tree. This is correct. This puzzle illustrates 
that the third dimension can easily confuse your normal reference \o reality if not 
appearing in conventional situations or from conventional angles. This weak part of 
human perception can be used to amplify cognitive uncertainties which are very 
important in thinking, learning, decision making and design. Just like perceptual tasks, 
cognition needs exploratory situations where we can discover the constancy in our 
relation to reality by experiments. For instance it is a good practice to imagine cutting 
planes if you like to become aware of complex shapes. 




Wire-frame 




Scale 

DistanceljSsIi 





® Object color Ought color 
Red 





fiLathe (MicroLathe) is an easy to use modelling tool 
that allows you to create three dimensional objects 
using the metaphor of the carpenter's lathe. The 
number of slices may be chosen and one of 
the display methods as well, the wire-frame is the 
most rudimentary one. Hidden, constant and Gouraud 
give an increasing level of spatial and light-reflective 
realism. 
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The different outlooks of the lathing procedure can immediately be viewed and 
controlled for certain conditions of light and rotation. The lathing metaphor enables the 
user to explore the relation between lathing curve and its final effect in 3D space. It 
illustrates perfectly the functionality of CADD technology (Computer Assisted Draft 
and Design). It is not only effective for the designer of visuals but especially adequate 
for students who use it as an exploratory tool for spatial awareness. 



Conclusions 



This chapter has raised some didactical opportunities which may benefit from graphical 
and spatial manipulations by the student. Browsing through visual images in 
interactive learning material until now is restricted to simple actions like panning and 
zooming like present in current CAD and drawing packages. Visual representations in 
computer systems allow the user to explore spatial perspectives in order to acquire a 
better understanding of physical objects, machinery and constructions like architecture 
and design 
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Before flexible and meaningful visual transformations become possible it is important 
that pictures are vectorized along at least three dimensions The fourth dimension (time 
axis) becomes important if it concerns an animated process, like important in 
explaining the four strokes of a combustive engine. 

Adding semantic relations between the components in a visualized constellation like an 
engine, electronic device, or an organic body can will even allow the student to explore 
its functionality as he/she changes the position, the size or even the shape of elements, 
after which the consequences in functionality will be displayed, like e.g. the 
temperature, current, concentration, speed, friction etc. 

The ultimate benefit from interactive visuals might even go beyond the spatial and 
mechanistic dentation. The given examples of 'impossible figures' as carriers for 
concept maps snow that spatial conflicts in pictures can be used to elicit pre cognitive 
ideas and hypotheses from the student at the beginning of learning sessions in order to 
provoke an optimal receptive mind for new ideas. Crucial issue in visualisation for 
learning purposes is the possibility to manipulate, change, combine by the student while 
the consequences can be seen immediately from the same picture. 
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Functions and Design of Video Components in Multi-Media Applications: a Review 
Plon W. Verhagen 



Introduction 

Designers of interactive multi-media applications face the decision when and how to be 
presented information has to take the form of text, graphics, animation, video stills, 
moving video, sound, or any combination of the former. This contribution focusses on 
design decisions with respect to the use of live or animated video segments with or 
without sound. First, functions and utilization patterns of video as a component of 
multimedia systems are presented. Next, research on audiovisual design is reviewed with 
respect to camera factors, audio factors, optical effect and special effects, pacing and 
rhythm, animation, picture complexity, and using two channels (visual and audio). 

Video functions 

Video functions may be classified according to content-driven or logistic criteria which 
may both be used to support decisions to select video as a component of an instructional 
application. Figure 1 provides a list that is based on Verhagen (1992, 1993). The 
functions are: 

Content-r elated use of photographic pictures or moving video (showing people and 
objects). This function concerns the message-specific need for a certain mode of 
presentation (if 1 require a visual, audio will not do; and if I have to show motion, 
video has to be preferred over still pictures). In certain fields, such as medicin, biology, 
and geography, there are many content elements for which it is self-evident that they 
need these kinds of visuals. It has to be noted, however, that in many cases message 
characteristics are not unequivocally prescriptive with respect to communication mode. 
Visual referents in memory may for instance inhibit the need for pictures, if they 
sufficiently support mental imagery in the context of verbal messages. Figure 1 shows 
that content relatedness plays a role in several of the other listed video functions. 

Depicting the invisible or the non-accessible. This concerns presentation options that 
cannot exist without certain media techniques. Examples are: slow motion, time- lapse 
photography, photography with invisible beams (like Rontgen or infrared photography), 
use of telescopes and microscopes, and animation. Here the "human size" with respect 
to perception in space and time is an important factor. Some experiences cannot 
happen if events could not be speeded up, slowed down, scaled up or scaled down to 
manageable proportions by use of media. 

Logistic use. Straightforward applications concern uses of media to solve problems of 
place, time, and identical repetition of messages. (In most cases, it is more feasible to 
show a slide of the Vesuvius than to visit this volcano). And, substantial effort to 
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develop an audiovisual programme which contains a refined discourse about a certain 
subject is a one-time investment. (The programme can be repeated infinitely with 
constant quality.) Logistic arguments apply also in the cases of several other functions, 
as is represented in Figure 1. 



Vid^o functions: 


j 

Purpose: 

Content-related Logistic 


Content- related use of photographic pictures or moving 
video 


* 


Depicting the invisible or the non-accessible. 


* 


Logistic use. 


* 


Cognitive help (visualizations as tools for thinking). 


• 


Modelling of psychomotor skills. 


• * 


Video feedback 




Providing observation materials. 


* • 


Psychologically realistic simulation. 


* 


Video to present gifted teachers. 


* 


Video design for emotional involvement. 


* 
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Cognitive help (visualizations as tools for thinking). Visualizations may offer new 
perspectives on knowledge or extend existing knowledge in the way that Bohr's Atom 
Model shaped thinking about how an atom is conceived. Bohr's model provided a way 
for insightful communication about the elementary decomposition of matter: a message 
for which visual media characteristics could be exploited succesfully. Similar reasoning 
applies, for instance, to the insightful animations (with narration) that demonstrate the 
essence of the Lorenz transformation or Einstein's Special Theory of Relativity. In 
general, schematizing is the keyword hsre. 

Modelling of psychomotor skills. Video is in many cases an appropriate means to 
demonstrate motor behavior. 

Video feedback. Video may be used to record student behaviour for feedback purposes. 
This may support the evaluation of psychomotor performances, feedback during 
microteaching sessions or interview training, and other. 

Providing observation materials. Video registrations (of group discussions, of children 
playing, of animal behaviour, etc.) may provide opportunities to train observation skills 
and to learn to analyse behavioural patterns. 

Psychologically realistic simulation. Social interaction may be simulated by having 
human models on for instance — videodisc who react to decisions of students by 
talking directly to the students from the video screen. Well-known examples are video- 
based management games. 

Video to present gifted teachers. In fact a variant of logistic use, this video function 
provides the possibility to repeatedly enjoy the presentation of outstanding teachers who 
demonstrate their skill in handling subject matter. 

Video design for emotional involvement. In this case, aesthetic value, emotional value, 
and use of drama are at stake. Factors regarding pleasure, goodness, beauty, interest, 
and complexity can be identified as components of aesthetic value. These factors play 
a role in determining the attention of learners and their perception of the relevance of a 
learning task which is a component of their motivation. The depicted objects may have 
emotional qualities of their own. Wounded victims of war or hungry people of Africa 
may evoke emotional responses that may have an impact for certain types of learning. 
Responses of anger, joy, curiousity, desire for justice, and other, may also be stimulated 
by appropriate use of drama. The filmic form of video offers excellent opportunities to 
exploit these possibilities. Recorded material has thereby the advantage of the carefully 
planned impact that results from skillful design and production. 

Basic approaches for utilization 

The above-mentioned video functions represent one set of variables to consider when 
designing practical application. Next to this, several basic utilization patterns may guide 
the designers: 



63 



Text primacy (symbolic first). Here, the basic question is: "Can printed material (text 
and pictures) do the job?" This starting point leads in many cases to cost-effective 
solutions. In cases where moving video is indispensible, this approach may be 
maintained by using barcode-controlled videodiscs as an adjunct to the written material. 
The approach is feasible for good readers with sufficient learning skills concerning the 
subject matter involved. 

Visualization as starting point (iconic first). Here, the basic question is: "Can we show 
what we mean?" This starting point leads in many cases to a look-and-tell format on 
the basis of video presentations. It results in concrete presentations that may very well 
suit rather concrete subject matter (or it may enlighten abstract subject matter by 
visualization). It is feasible for group instruction as well as for individual study, may 
help poor readers, and may involve students that are not very motivated to read. 

Between-channel redundancy as reinforcement strategy . Using two sensory channels 
increases the likelihood that messages arrive. This phenomenon may be used by 
developing educational materials in such a way that sight and sound are simultaneously 
used to present semantically overlapping information. The simplest (and most 
redundant) form is that a voice presents texts that can be read from paper or from a 
computer screen at the same time. Striving for semantically overlapping information 
leads often to audiovisual presentations in which schematizing is frequently used to 
support abstract subject matter with appropriate images. Between-channel redundancy 
as reinforcement strategy is appropriate for complex subject matter and/or poor learners 
as long as the total amount of information does not exceed the information processing 
capacities of the learners. 

Orchestration for learning (theater of life). This concerns the use of the video function 
for emotional involvement. Creative communication solutions may attract and maintain 
attention, establishing learning situations that are entertaining, challenging, or have an 
emotional impact for other reasons. This technique is appropriate if motivation to learn 
is considered a problem, for instance in distant learning situations where the delivery of 
instruction takes place by means of television and has to compete with other television 
channels. 

Audiovisual archive as a starting point (providing a computerized information 
environment). In this case the database capabilities of multimedia are exploited. This 
may be done for the purpose of knowledge acquisition in an eductional setting, where 
personal development is paramount. Free navigation through the information space is 
here the starting point albeit that the space available may be limited for didactical 
reasons to prevent novice learners from being overwhelmed by the difficulty level of 
encountered information or from getting lost in the possibly vast amount of information 
contained in the system. In all cases, adequate help functions should be available. For 
experienced users, playing around in the information environment offers ways to extent 
and deepen knowledge by enriching existing schemata with new facts or new 
combinations of facts. 
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An other application concerns problem solving on an individual basis with the multi- 
media system as the main source of information. In that case two more options are 
there: (a) the problem-solving activity is guided by the system (it is incorporated into a 
detailed planned teachin g method), or (b) the system is an open information 
environment in which the user has to find his or her own way. The level of support by 
the system may be adjusted to the needs of the user, either automatically (with an on- * 
line help system that detects ineffective conduct of the user), or on request (when the 
user decides to switch from the browse mode to M challange me", n teach me", or 
"question me"). Applications may further take the form of interactive encyclopedia, 
interactive maintenance manuals, etc. 

Interactivity as the star ting point. This is the case when learning outcomes are desired 
for which it is considered to be necessary that practice, testing, and feedback are 
organized in interaction with the computer system. The main instruction functions are: 
orienting on the learning task (presenting subject matter, presenting content-specific 
ways of thinking, presenting new methods and techniques to operate on the subject 
matter, demonstrating working methods and problem-solving approaches), practice 
(providing opportunities to practice, providing feedback), testing and general feedback. 
For the organization of multimedia applications to serve these instruction functions, 
common computer-based instruction formats may be used, such as drill-and-practice, 
tutorial, and simulation, extended with the presentation richness of multimedia. 



The video functions and the basic utilization patterns are not independent of each other. 
Figure 2 shows how the different functions relate. 
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Basic utilization patterns 


Supported by the following video functions 


Text primacy 


- Limited supplementary use of content-related video 


Visualization as starting point 


• Content-related use of video 

- Depicting the invisible or the non-accessible 

- Logistic use of video 

- Cognitive help 

- Modelling of psychomotor skills 

- Providing observation materials 

- Video design for emotional involvement 


Between-channel redundancy as 
reinforcement strategy 


- All functions with proper audio layers added 


Orchestration for learning 


- Video design for emotional involvement 


Audiovisual archive as a starting 
point 


- Content-related use of video 

- Depicting the invisible or the non-accessible 

- Logistic use of video 

- Cognitive help 

- Modelling of psychomotor skills 

- Providing observation materials 

- Video to present gifted teachers 


Interactivity as a starting point 


- Modelling of psychomotor skills 

- Video feedback 

- Providing observation materials 

- Psychologically realistic simulations 



Figure 2: Video functions from Figure 1 in relation to basic utilization patterns 



The video functions and the basic utilization patterns form a limited set of factors to 
consider when selecting or developing multimedia applications. For instruction, proper 
handling of learning objectives, content analysis, and learner characterisitcs require 
thorough knowledge and skills with respect to the knowledge domain of the subject matter 
and of methods and theories of instruction, and knowledge and skills with respect to 
methods for multimedia design and production. These subjects are beyond the scope of 
the present paper. This contribution focusses on results from audiovisual research that 
relate to detailed design decisions for developing video presentations. This concerns 
primarily the utilization patterns "visualization as starting point 4 ', "between-channel 
redundancy as reinforcement strategy", and "orchestration for learning". 
Much of the audiovisual research reported is carried out with linear video programmes. 
The findings arc, however, probably to a large extent applicable to components for 
multimedia solutions. 



Results of audiovisual research on presentation variables 



Presentation variables are actually production variables that obtain their values during the 
production of video material. Below the following variables are discussed (derived from a 
list by Coldevin, 1981, p. 87): 
(aycamera factors (angle & shot); 

(b) "setting; 

(c) "audio factors;" 

(d) "special effects; 
(eypacing and rhythm. 

In addition, a brief passage is devoted to animation. The variables are listed in Figure 3 
in which they are positioned relative to the production phase in which the pertinent design 
decisions are taken. 

Camera Factors 

As far as research is concerned, the effects of camera factors on learning seem to be 
limited and results are often conflicting. Dwyer (1978, p. 168), for instance, refers to 
Roshal (1949) and McCoy (1955) to conclude that it is advisable to show a performance 
on the screen the way the learner would see it if he were doing the job himself. Coldevin 
(1981, p. 88), however, describes an experiment by Grant and Merrill (1963) that shows 
for relatively complex skills (and perhaps for most task-oriented productions) that the 
viewing angle of the demonstrator should not be used rather than the perspective of the 
student viewer who is watching the demonstration. 

Research by Salomon (1974) seemed to show that certain camera factors — such as 
zooming in — can positively supplant visual information that young viewers must fill in 
during cuts between shots. It has, however, been shown that this supplanting effect can be 
demonstrated by other means (Bovy, 1983). 

This effect is consistent with the fact that different authors often present different solutions 
to serve certain communication function by camera factors. Zooming-in for instance, is 
classified by Morrison (1979, p. 29) as a device for: focusing attention, relating parts to a 
whole, emphasizing one aspect, and showing spatial relationships. All these function can 
be accomplished by other means. Morrison himself says that the cut can be used to (re- 
)focus attention and to relate parts to the whole. 

But there seems no reason to exclude the use of cuts for emphasizing and for showing 
spatial relationships. Moreover, for all these functions, the superimposidon of optical 
markers can also be used (such as arrows, animation, encircling; see for instance Dwyer, 
1978, p. 160). Lumsdaine and Suizer (1951) had earlier shown the effectiveness of 
devices of this kind for directing perception in films. Boeckmann, Nessmann, an A I 
Petermandl (1988) confirmed this effect as part of an experiment in which subjects had to 
watch a video programme with the task to notice objects and procedures that are forbidden 
in a professional kitchen for hygienic reasons. In the experimental version of the 
programme, three things were made to stand out in a long shot by means of 
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Design 
decisions in 
production 
phase: 


Presentation 
variable 


Purpose 


Examples 


scripting and 
shooting 


camera angle & 
shot 


framing of to be 
depicted objects 


close-up to focus attention, wide- 
angle shot to establish a visual 
context for a beginning scene, etc. 




audio (narration, 
live sound, and 
music) 


supporting the 
visual layer or 
(in case of text) 
mainstream of 
factual 
information 


narration to explain what the 
picture shows, live sound to 
demonstrate how a MIG-welding 
arc should sound, music to support 
a story-relevant mood. 


scripting and 
editing (post 
production) 


optical 

effects/special 
effects 


getting attention, 
relating scenes, 
symbolizing 
transitions in 
place or time 


wiping to connect mutually 
supportive or opposing visual 
situations, lap dissolve to move to 
a scene in the past, fade out/ fade 
in to suggest the passing of time, 
etc. 




pacing & rhythm 


providing a 
presentation 
rate that gets 
and maintains 
thougthful 
attention (by 
complying with 
the spectators 
preferred state 
of cognitive 
naco) 


shorting time by inserting only 
action-laden shots, edited for 
narrative continuity at an attention- 
demanding pace. 


scripting, 
shooting, 
editing 


animation 


attention gaining 
and visualization 


instructional visualizations without 
distracting cetails, focussing 
attention by animated pointing 
devices 



Figure 3: Selected production variables for video materials 



superimposed rings (flowers, cigarettes, and domestic animals). The results of that 
experiment show that: "the most impressive increase in recall was observed in the case of 
the point concerning flowers, these being difficult to see in the background of the picture 
(38% of the students who had seen the experimental version mentioned flowers in 
comparison with none of the students who had seen the original version)" (Boeckmann, 
Nessmann, & Petermandl, 1988, p. 118). 



ERLC 
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This supports the notion that camera factors do not have definite meanings that can be 
used as invariable building blocks in message design. Just like the ideas about editing that 
are put forward by Reisz and Millar (1981), the use of camera factors should be guided bv 
the nature and order of the message components that are to be conveyed. Studying the 
effects of single shots seems not to be very useful in that respect. It is more that 
sequences of shots should establish adequate patterns to cover the scenes to be depicted. 
This is exactly what media handbooks offer to support the development of solutions for 
particular design problems. Arijon (1976) for instance presents hundreds of model 
solutions for a wide range of camera problems, varying from staging straightforward two- 
person dialogues to staging complicated mass scenes or finding solutions in confined areas 
such as the interior of an aeroplane. It seems thus warranted to approach the treatment of 
the camera from the perspective of staging a message as a coherent series of shots. 
Consistent authorship may thereby lead to a style that establishes clarity and appeals the 
audience. The following research results are consistent with this view: 
Low angle shots can increase the perceived potency c a presenter, but the framing of 
these shots in the narrative structure has then to support this intention (Coldevin, 1981 p 
88). 

In video programmes, interest levels can be increased with medium close-uos or with 
close-up shots (p. 88). Boeckmann, Nessmann, and Petermandl (1988, p. 1 10), however, 
concluded on the basis of their research that recall was not related to type cf shot (close- 
ups, long shots, or intermediate shots). 

These results illustrate that the function of shot type is dependent on the circumstances 
under which it is used. The camera treatment has thus to be regarded as a flexible 
component of audiovisual message design, whereby research-based rules to predict good 
results are weak ur absent. Good results are dependent on the professionalism of 
responsible team members in a media project. Rules from practice such as can be found 
in media handbooks are useful to support design decisions (see, for instance, Arijon, 1976, 
or Millerson, 1985). 

Audio factors 

Audio factors are narration or dialogue (in the case of speaking characters), effects (the 
'natural' sounds of the setting that is depicted on the screen (effects may be recorded live 
or be produced artificially), and music. 

For instruction, off-screen commentary is often used. According to Dwyer (1978, p. 169), 
the number of words per minute of film in the commentary has a definite effect on 
learning. He refers to Zuckermann (1949a) and Jaspen (1950a, 1950b) to conclude that 
care should be taken not to "pack" the sound track. Schmidt (1974, p. 332) found in his 
analysis of the design of 20 outstanding instructional films that the majority of these films 
had an average narration rate of 140 or fewer words per minute. Dwyer (1978) advises 
that: 

"Too much or too little talking in words per minute of film has been found to detract from the 
teaching effectiveness of a film. The optimum word rate is about 100 words for each minute of 
film." (p. 172.) 

The normal rate of speech ranges from 150 to 200 words per minute (Rossiter, 1971). 
This leads to a rule of thumb that it is a normal value for narration to take about two- 
thirds of the total presentation time of an instructional film. 
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Schmidt (1974, p. 332) also found that the majority of the outstanding instructional films 
he analysed: (a) did not use a style of narration that talked down or lectured to the 
audience; (b) did simplify the message as much as possible; and (c) used the active form 
of sentence structure. That 'simplification of the message' was a factor most probably 
related to the fact that the analysed films were not designed for specific audiences. Still, 
the message of Schmidt's observations is that narration should take the audience seriously, 
present in a clear, concise, and unambigious way, and talk directly to the audience if the 
subject allows it. 

About the other factors -- effects and music — research results are very limited. Fleming 
and Levie derived one principle with respect to concept learning; that if a concept is 
basically temporal - such as rhythm or time or sequence or frequency, or like poetry or 
music or speech - then audition is appropriate (Fleming & Levie, 1978, p. 48). Research 
about sound effects was not mentioned further by the authors reviewed. The informational 
and emotional nature of this kind of sound seems to be considered as self-evident. 
Research about music in instructional audiovisual programmes is scarce. When Coldevin 
(1981) prepared his review, he could refer to only one study (by Baggaley, Ferguson & 
Brooks, 1980). In that study different types of music for opening or closing a programme 
were subject to experimentation with unconclusive results due to methodological problems 
(Coldevin, 1981, p. 89). With respect to music, Jaspers (1991a) reinforced the conclusions 
made by Zuckermann (1949b) that music has informational, emotional, and 
conceptual/integrative functions. Leit-motivs and other musical effects may help to 
structure a presentation and inform the listeners about what is going to happen. Tempo, 
modality, and rhythm may be used to evoke emotional responses. According to Jaspers, 
proper use of music may support instructional messages but -- unless the topic is about 
music - there is no need to use it "when the purpose is transmission of referential 
information and cognitive information processing" (Jaspers, 1991a, p. 50). Music may, 
however, facilitate learning by giving students occasions for identifications and empathy. 
However, as Jaspers puts it: "well-chosen music does not guarantee to create the supposed 
effect, but it is highly probable that ill-chosen music will do much harm to the learning 
process" (Jaspers, 1991a, p. 50). 

Given the limited results from research, the main conclusion has to be that the treatment 
of sound effects and music has to lean on rules from practice. Insuring ideas can, for 
instance, be found in Millerson (1985, p. 353-362). 

Optical effects/special effects 

The review by Dwyer U978) as well as more recent research by Boeckmann, Nessmann, 
and Petermandi (1988) show that optical effects such as fades, wipes, and dissolves do not 
increase the instructional effectivity of films or video programmes compared to straight 
cuts to connect subsequent shots. Ginsburg, Bartels, KJeingunther, and Droege (1988) 
found that with special effects, the situation may even be worse. They carried out 
experiments with a programme that contained highly abstract visual effects and found that 
these effects diminish recall. Most probably, the observation of Boeckmann, Nessmann, 
and Petermandi (1988) applies that formal elements - such as a fades or wipes - that do 
not relate to the programme content have no influence on recall (which according to 
Ginsburg et. al. may even be negative in the case of special effects). Boeckmann et. al. 
demonstrated that formal elements that played an explicit role to define a message proved 
to influence recall. The conclusion seems to be that deliberate content-related use of 
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optica] effects or special effects can be feasible, depending on their functionality that 
affects the quality of a message. As with the use of the other presentation variables, the 
message design has to be content driven. This is, however, not to deny that form 
influences content. The eventual meaning of a message is determined by the combination 
of the depicted and the manner of depiction. It may be stressed that for instructional 
purposes, form and content should be balanced with the intention to reinforce the 
substance of the instructional message. 

Pacing & rhythm 

A question is, in what way does the pace of presentation in films or video programmes 
have to be controlled to account for cognitive pace (the rate of mental processing of 
incoming data)? One principle is that: "the rate of development of a film should be slow 
enough to permit learners to grasp the material as it is shown (Jaspen, 1950a; Ash & 
Jaspen, 1953)" (Dwyer, 1978, p. 168). 

According to Dwyer this means that the rate must be slow rather than fast. The 
outstanding instructional films that were analysed by Schmidt (1974): 

"... had a rate of development that was slow enough for the viewer to grasp the material as it 
was shown" (p. 332). "... had a slowing of the rate of development at points at which it is 
necessary for the viewer to change attention from one source of information to another" (p. 
332). "... had an average shot length of about 10 seconds" (p. 334). 
The 10 seconds are of the order of magnitude of the time that Simon (1974) has put 
forward as the time it takes to fix a chunk of information in long-term memory. The 
same 10 seconds seem also to be candidate for a characteristic parameter in a study by 
Leahy (1983). Leahy studied the relationship between visual realism and information 
processing time in viewing projected visuals. His results, with respect to manipulating 
projection time to promote more efficient visual comprehension, did not yield clear advice. 
He got, however, the impression that the fact that he did find significant differences 
between projection times of 5 and 10 seconds and between 5 and 15 seconds, but not 
between 10 and 15 seconds, that after 10 seconds a ceiling affect might have occurred. 
Whether this means in general that a mean cutting rate of six shots per minute yields an 
adequate pace of an instructional film can, however, not be concluded. 
Coldevin (1981) referred to a study by Schlater (1970) that showed that cutting rate should 
not be too slow: 

"Schlater (1970) produced five videotapes in which the visual changed ranged from one 
every 30 seconds in the first treatment to nine per 30-second interval in treatment five. 
Three types of learning tests were administered: pictorial video (visual sketches), verbal 
video (verbal descriptions of pictorial information) and audio channel information. 
Testing results showed that verbal video comprehension increased directly with the 
increase in the rate of visual changes (levelling off at seven visuals per 30-second 
interval), but the opposite effect was noted in pictorial video recall. As might be 
expected, no significant differences were found between the treatments on recall of 
audio channel information." (Coldevin, 1981, p. 91) 

It thus seems that under most circumstances a cutting rate that corresponds with a mean 

shot length of about 4.5 seconds may be feasable. 

A further study by Vincent, Ash, and Greenhill (1949) studied the relationship of length 
and fact frequency to effectiveness of instructional films. "Fact frequency" may thereby 
be conceived as a factor that is directly related to the cognitive pace mentioned earlier. 
The procedure in the study was as follows: 





"Four experimental film versions dealing with the causes and manifestations of the weather were 
made up from a senes of Navy training films on aerology. The Long Heavy version ran 29 
minutes and contained 224 facts; the Long Light version also ran 29 minutes but contained 1 12 
facts. The Short Heavy version ran 14 minutes and contained 1 12 facts; the Short Light version 
ran 14 minutes but contained 56 facts. The total number of words in each pair of equal length 
kept constant by the use of repetitions, prefatory statements, and other Tiller' material which did 
not add new facts. The four experimental versions were shown to four groups in each of three 
different populations. High School students (12th grade). Air Force basic trainees, and College 
students. In each population a fifth control group did not see the film. All groups took the 
same 1 36 item multiple-choice test. The High School and Air Force groups took the lest again 
after delays of four weeks and seven weeks respectively." (Vincent, et al., 1949, p. 1.) 
The pace in the heavy versions was thus about one retainable fact per 7.5 seconds. In the 
light versions about one retainable fact per 15 seconds was presented. The reported 
narration rate was about 124 syllables per minute. The results showed that significant 
learning occured: 

"Every group that saw experimental film earned a substantially higher score than the control 
group which did not see a film. ... For the High School sample the Short Heavy version 
seemed to be most effective, for the Air Force and College samples the Long Light version 
seemed to be most effective. At the end of the delayed recall period all differences among the 
versions were much smaller than they had been on the immediate retention test, and most of 
them were not significant." (Vincent, et al., 1949, p. I) 
Test scores on immediate recall ranged from slightly more than 40% to about 50%, in the 
experimental groups; delayed test scores were roughly speaking 20% lower. Differences 
between groups were of the order of magnitude of 0 to 5%. Vincent, et al. point at the 
fact that the films were rather difficult for the populations used. Their overall conclusion 
is that: 

"It seems clear from the data that packing more and more information into a film yields only 
very slight increments in total measured learning (underlining omitted, pwv). In no case did the 
Long Heavy film group learn anything approaching twice as much as the Short Heavy or Long 
Light groups, nor did the latter learn twice as much as the Short Light group." (Vincent, et al., 
1949. p. 2) 

With respect to the audio channel, the narration rate that was discussed under "audio 
factors" also plays a role in pacing a programme. The narration may "drive" the 
processing of visuals more when the two information channels are sufficiently 
correspondent (Grimes, 1990, see also the passage about two channels below). Clear 
research-based rules on how to proceed when deciding on cutting rate and narration- 
related pacing, however, do not exist. 

Animation 

Animation is traditionally a representation technique from the film and video world. It 
varies from simple stop-motion formats in which message components are stepwise 
developed (in instructional situations usually synchronous to narration) to detailed cartoons 
in full motion. The technical developments of computers have led to beginning 
possibilities to integrate animation into computer-based instruction. 

Rieber (1990) posits that in computer-based instruction animation has been used to fulfill 
or assist one of three functions: attention-gaining, presentation, and practice. Rieber sees 
reason to advise that animation should be incorporated only when its attributes are 
congruent to the learning task. These attributes are: visualization, motion, and trajectory 
(= the path of travel of an animated object). 
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In the audiovisual world, the situation is not much better as far as research is concerned. 
The usefulness of animation techniques seems, however, to be accepted as rather self- 
evident. Animation is considered to be a common presentation option that can be put into 
operation whenever its application seems feasable. The functions mentioned by Rieber are 
then also at stake, as are even simpler applications such as providing cueing devices like 
superimposed boxes and arrows. 

An important feature of animation is that animated visualizations can be strictly to the 
point, without any distracting detail. Wember (1976), for instance, did an experiment with 
a video report about the political and societal conflict* in Northern Ireland. In this report 
one sequence was about the historical causes of these conflicts. The images that 
accompanied this sequence were originally scenes of a town (street scenes, graffity on 
walls, etc.). Wember replaced these images by an animated scheme of the historical 
development (on the basis of a highly stylized map of the British isles) that proceeded 
synchronous to the narration. In this way he was able to demonstrate a dramatic gain in 
recall and comprehension scores with respect to the intended message (from an initial 
score of 30% correct with the original version to a score of 80% correct with the version 
in which the live scenes were replaced by the animation). Wember suggested that the 
effect of the animation was especially convincing because of the fact that the supported 
message was abstract. It is generally difficult to find realistic images that support abstract 
messages that are contained in the commentary. Animated schematic drawings may in 
that case offer the solution and add to the impact of the presentation. 

Aspects of information load 

Next to presentation variables, the communicative features of video presentations depend 
on the cognitive load they cause in the mind of the viewers. Picture complexity should 
not be beyond what a learner is able to understand and the audio and video channel should 
not be thus information loaded that they compete for the attention of their audience. In 
this section, relevant research is reviewed leading to a list of considerations that designers 
should keep in mind when working on video segments (Figure 4). 

Picture complexity 

"Picture complexity" refers to the syntactic, semantic and pragmatic difficulty of 
comprehending a picture. Other factors also can determine complexity: 

- Density of information in the sense of both the number of picture elements or objects 
and of the degree of detail with which each element is depicted (Peeck, 1987, p. 137). 

- Syntactic cues such as overlap of objects to augment the depiction of depth. 

- Semantic clarity, which refers to the esse with which picture elements can be called 
by name. 

- Pragmatic clarity, which concerns the kind and amount of existing knowledge that is 
needed to interpret the picture. 

Each of these factors have been subject to research with respect to a diversity of variables 
that can be identified to play a role. Overviews can be found by Goldsmith (1984, 1987), 
Peeck (1987) and Leahy (1983). Here, only some general issues are discussed with 
respect to the use of realistic video images as well as the use of schematic representations. 
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The effectiveness of realistic pictures for learning has been a matter of debate for a long 
time: 

"An ongoing controversy exists in the literature regarding the role of realism in 
visualized instructional materials. A number of theorists, writing during the 1940s and 
early 1950s, suggested the existence of a realism continuum along which all materials 
could be scaled in relation to their instructional effectiveness. ... 
However, more recent researchers ... have disagreed with these theories on the grounds 
that moie realistic materials will tend to present too many irrelevant cues which will 
interfere with learning. Miller (1957) suggests that too many irrelevant cues may 
actually compete with one another for attention and subsequent information storage. ... 
for some educational objectives, simplified or less realistic visual instructional materials 
are more effective in falicitating student achievement than more complex photographic 
or detailed drawings of the same object. Dwyer found strong evidence to indicate that 
the most realistic visual illustrations were not necessarily the most effective 
instructionally. " (Parkhurst, 1975, p. 175-176) 
Heuvelman (1989, p. 38-40) discussed the results of several studies that suggest that for 
instruction, schematic visualiiations yield better learning results and are thus preferred. 
Dwyer (1972) provided arguments why realistic images may hinder learning in the case of 
externally paced presentations (as in video programmes): 

"The initial impact of the excessive realistic detail may be sufficiently strong to detract attention 
from relevant and important learning cues. ... The process of indentification and discrimination 
is time consuming; the more intricate the visual stimuli, the longer it takes for the student to 
identify and absorb the intended information." (Dwyer, 1972, p. 90-91.) 
The research of Dwyer and others focused on the dimension of realism-schematism. This 
focus may, however, distract from the real instructional issue. The question is whether the 
instructional goals need the presentation of concrete referents (to teach concrete concepts, 
to demonstrate psychomotor procedures, to embed instruction in a familiar context; or any 
other application for which it is necessary to provide real-world schemata that require 
"gestalts" with proper fidelity) or whether schematic visualizations are better to articulate 
the instructionally relevant features (visualizing structures, relations, and principles). 

In the case that real-world schemata are indicated, results other than those in favour of 
schematic visuals may be found. Indeed, Leahy, in his research with fixed-pace 
presentation of visuals, found that complex high-realism visuals promoted a significant 
increase in visual comprehension over low-realism visuals Leahy used colored visuals of 
high complexity on two levels cf realism that were of relevant instructional content to 
social studies, with emphasis on object-interrelationships and conceptual content, rather 
than mere object depiction. Subjects were public high school sophomores. The properties 
of the high-realism visuals that seem to be responsible for his results may, according to 
Leahy, be "a more meaningful, coherent, or experientially recognizable pattern of object- 
interrelationships" in these pictures (Leahy, 1983, p. 107). 

With respect to picture complexity, Leahy (1983, p. Ill) listed as one of the implications 
of his study that high-realism visuals are apparently more efficient with respect to the 
recall and comprehension of presence and appearance of information and of the 
interrelationship of objects. 

Next to this, another argument for some degree of complexity is that complexity invokes 
interest, as is apparent from one of the research-based principles of Fleming and Levie: 
"attention is drawn and held by complexity, providing the complexity does not exceed the 
perceivers' cognitive capacities" (Fleming & Levie, 1978, p. 22). 
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A last point is that picture quality has to be such that the images can be appreciated 
without technical imperfections. Fleming and Levie formulated one of their principles for 

instructional message design: "the better an object or event is perceived the more 

feasible and reliable will be further cognitive processes, e.g., memory, concept formation, 
problem solving, creativity, and attitude change" (Fleming & Levie, 1978, p. 88). 
The use of professional video formats and professional camera treatment and editing 
facilities seem thus to be indicated in order to produce effective video materials. 

Using two channels 

In audiovisual presentations, the visual channel and the audio channel are used 
simultaneously to convey messages. Several researchers have tried to find how these two 
channels interact. Jaspers (1991b) reviewed research regarding the relationship between 
sound and image. His findings, supplemented with findings of Dwyer (1978) and Kozma 
(1991), are that: 

"Visual images are reflections of physical presence" (Arnheim, 1986), presenting 
"concreteness" (Mast, 1977) that is represented by the surface of the depicted objects and 
events. Sounds are, in most cases, attributes of visual objects (Metz, 1975), with the 
exception of sound objects such as a hiss, thunder, music (Jaspers, 1991b, p. 164). But, 
according to Metz, voices may also be aura! objects as far as they have meaning of their 
own, apart from the words spoken (recognizing an off-screen voice may be important to 
value the meaning of its speech). For instructional applications this is, however, a minor 
point in most cases or even not relevant. 

Language communicates thoughts (Arnheim, 1986); it presents abstractions (Mast, 1977). 
This counts whether or not language is presented visually (in print or on a video screen) 
or as an audible sound. 

When used in combination, most researchers conclude that an audio-visual combination 
can be more effective than audio alone or video alone (Hoban & Van Ormer, 1950, 
reprinted in 1970; Coldevin, 1981; Nugent, 1982). Kozma (1991, p. 191) reviewed more 
than ten studies in which separate audio and visual presentations of a video programme 
were compared and found that: "in none of these studies did the combination of audio and 
visual information result in lower recall than recall from either source alone. In most of 
these studies, the combined use of visual and auditory symbol systems resulted in more 
recall than visual-only and audio-only presentations." Although it is thus often the case, it 
not always evident, that audio plus vision is better (Hsia, 1968). The effects of the 
combination appears to be relative to factors of learner and material (Hoban & Van 
Ormer, 1950; Johnston, 1987; Barton, & Dwyer, 1987). 

Often, dominance of vision over audio is reported. "The overall influence of the motion 
picture is thought to be primarily in the picture and secondarily in the accompanying 
language" (Dwyer, 1978, p. 172). Visually presented information would be retained better 
than aural information. But. for instance, Hsia (1968) concluded that "the question of 
supremacy of the auditory or visual channel is controversal. Some studies indicate that 
seeing is better than hearing, others show the reverse" (Jaspers, 1991b, p. 164-165). 
These results seem to support one of the principles for improving visual learning that was 
proposed by Dwyer: 

"Both audio and visual elements of films are effective channels of communication. 
Neither channel is consistently better than the other. Each channel is uniquely capable 
of conveying certain types of information and the two should be properly integrated." 
(Dwyer, 1978, p. 172.) 




The results also agree with one of the research-based principles of Fleming and Levie: 
"More learning can occur where information is received concurrently in two modahties, e.g., 
vision and audition or vision and touch, than where received in onlv one modahtv." (Fleming & 
Levie, 1978, p. 107) 

In instructional audiovisual programmes, sounds other than language or music are in most 
cases attributes of visual objects in the sense of Melz' (1975) assertion. (Instructionally 
relevant sound objects other than music do, however, exist. Examples are ticks that 
increase in speed to inform learners about the approach of the end of a period for a timed 
answer, and the different warning signals and other tones that are often used to give 
feedback on learner actions during learner-computer interaction. For audiovisual 
presentations, sound applications of these types are not very often used.) 

With sounds as attributes of visual images, the relationship between the visual images and 
language becomes the central issue for instructional audio-visual message design, in 
particular if the language consists mainly of off-screen narration, which is the case with 
the often used look-and-tell format. It has, however, to be noted that Boeckmann et. al. 
(1988, p. 110) found that in their experiments no influence on recall could be established 
in connection with the fact "whether the main source of sound was shown or not (e.g. 
whether the commentator was visible or a voice-over was used)". This suggests that 
conclusions with respect to off-screen narration may apply to presentations by a studio 
presenter as well. 

With respect to off-screen narration two variables are at stake: (a) the correspondence of 
the visually presented information and the commentary; and (b) the use of narration- 
supporting superimposed text on screen. Both are treated below. 

Correspondance of video images and audio narration 

The relationship between the information in the two channels determines whether the 
proper integration occurs as Dwyer has suggested. Some researchers have put forward the 
idea that "redundancy" and in particular "between-channel redundancy" plays a role. 
Jaspers (1991b, p. 168) describes that redundancy is formally defined as one minus the 
ratio of actual-to-maximum information transmission, but that the common use of the term 
is much more informal and refers to superfluous, extra, or repetitive information. Hsia 
(1971; 1977) has opted for optimal between-channel redundancy which he sees as the key 
to better communication. Between-channel redundancy in audiovisual communication 
occurs if information that is presented in in the visual channel matches the information 
that is at the same time presented in the audio channel. A methodological problem 
appears to be that it is not an easy matter to decide on the extent to which certain 
information can be conceived to be redundant. Pictures have, for instance, many cues that 
may be used by learners to connect new information to elements of their existing cognitive 
structure, while with audible text it is assumed that to be redundant to the picture only one 
perspective with respect to that picture is offered. This concerns the problem of the 
limited possibilities of translating information that is represented in one communication 
mode (for instance text) into a representation in another mode (for instance a 
visualization). In the past decades, a body of research and theory has developed with 
respect to sign systems (often also referred as symbol systems, where the term "symbol" is 
then used as a synonym of "sign"). Gross (1974, p. 60) speaks of primary modes of 
symbolic behaviour and distinguishes: (a) the linguistic, in which oral or written language 
is carrying the messages; (b) the social-gestural, in which facial expressions and body 
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language communicate messages between people; (c) the iconic, in which communication 
takes the form of visualizations; (d) the logico-mathematicai, in which the symbolic 
language of mathematics constitutes an independent way of communication; and (e) the 
musical, in which people express themselves with musical sounds. They are: "identified 
with (a) a range of objects and events or field of reference, (b) a distinctive memory- 
storage capacity, (c) a set of operations and transformations, and (d) specific principles of 
ordering which govern the formulation and communication of meaning, and (e) 
nontranslatability into other modes" (Gross, 1974, p, 61). With the last point, Gross 
means that information which is coded within one mode will not be capable of being fully 
recoded in terms of another. Nontranslatibility is thus a characteristic of a communication 
mode which indeed limits the possibilities for between-channel redundancy. It is 
therefore better to speak of more or less correspondence between pictures and sound, 
whereby strong correspondence resembles redundancy but is, in fact, not the same. This is 
here thought to be one of the reasons why results of experimental research fail to 
demonstrate that striving for redundancy yields results that contradict predictions on the 
basis of cue-summation theory (see fcr instance Severin, 1967), Cue-summation theory 
says that adding non-redundant cues in different channels enriches a message, thus 
offering more opportunities for learning (as long as the information processing system of 
the learner will not be overloaded), and redundant cues do not (because they add no new 
information). If after all alleged redundant information appears not to be as redundant as 
was assumed, improvement of learning that is reported on the basis of redundancy may 
very well be the consequence of cue-summation. A second reason why redundant cues 
may support learning despite of the predictions of cue-summation theory is, that in this 
theory the didactically beneficial effects of repetition are not taken into account. In 
practice, most probably, both reasons play a combined role. 

Grimes (1990) used a study of Drew and Grimes (1987) as starting point to study the 
influence of channel correspondence on recall of auditory or visually presented 
information. 

"Drew and Grimes found that in the optimum viewing condition — high channel correspondence 
- audio recall and story understanding were the highest, and video recall was the lowest. In the 
least optimum viewing condition — no channel correspondence — video recall was the highest, 
audio recall was the lowest, and story understanding was the worst." (Grimes, 1990, p. 16) 
It has to be noted that Grimes* research concerned television news, a format in which the main 
stream of information is traditionnally contained in the audio channel. Grimes reasoned that the 
results of Drew and Grimes are a consequence of limited memory and/or attentional capacities: 
"Channel correspondence probably affects the way in which attention is divided between the 
auditory and visual channels. Because attention is a limited commodity (Kahneman, 1973), 
more attentional capacity is probably required to encode and integrate in memory the two 
information channels when those two channels convey slightly to moderately different messages. 
With enough difference in messages (e.g., a medium-correspondence condition), attention's 
capacity threshold will probably be approached, maybe even exceeded, and viewers may try to 
concentrate on the auditorv channel — the who, what, when, where, why channel - at the 

expense of the less specific, and less useful, visual channel high channel correspondence 

should allow the viewer to monitor the auditory channel for the who, what, when, and where of 
the story and to shadow the visual channel for information that supplements the narration and 
can be integrated into the information supplied by it. Thus, attention to, and memory for, the 
visual channel should be greater in this condition than in the medium-correspondence condition. 




However, if the conflict between the two information channels is great enough (e.g., the no- 
correspondence condition), attention to the visual channel will override attention to the auditon 
channel." (Grimes, 1990, p. 16) 
This last idea is, according to Grimes, not only supported by the non-correspondence data 
from the Drew and Grimes study, but also by the work of Colavita (1974) and Posner. 
Nissen, and Klein (1976) who report that visual stimuli dominate auditors- stimuli when 
pictures and words have no relationship to one another (Grimes, 1990, p.' 17). Grimes 
carried out an experiment in which he focussed on the iocus of attention as it was split 
between the auditory and the visual information channels. "The purpose was to discover 
which channel, if either, received less attention due to diminishing channel 
correspondence" (p. 1 7). He used three versions of four television news stories as stimuli. 
Grimes operationalized "High Correspondence" by a "tight" picture-word match in the 
experimental material. When, for instance, the narration talked about a needle with the 
size of a screwdriver that is inserted into a herniated disk in the spine, the picture showed 
a needle, the size and shape of a screwdriver, about to be inserted into the spine of a 
patient. "Medium Correspondence" was operationalized by relating the two information 
channels thematically but with no further correspondence. The narrated sentence 
mentioned above (about the needle) was then accompanied by a video scene of a patient 
being wheeled into surgery. 

In the "Non-Correspondence" condition, the visual track consisted of a visual potpourri 
that did not match the audio. "For instance, as the narrator described how herniated disks 
are repaired, the viewer saw a barge loaded with garbage anchored in New York Harbor. 
When the shot changed, it showed celebrants of the most recent Chinese New Year, while 
the narration continued to discuss surgery" (p. 18). Story narrations were the same across 
all conditions. Attention to the visual or the auditory channel was measured with a 
reaction-time technique based on visual or tone probes. Half of the subjects were exposed 
to seven visual probes (NTSC colour bars during 33 milliseconds) nested within each 
story, and the other half were exposed to analogous tone probes (with a frequency of 10 
kiloHertz). In the technique subjects have to react to the visual or auditory signal each 
time that it appears by pushing a key as soon as possible. The reaction time is a measure 
of attention. The longer it takes, the more attention is assumed to be devoted to the 
channel within which the signal appears (due to the channel capacity that is occupied by 
the attention). Two recognition posttests were used: (a) a factual memory test with 53 
multiple-choice questions; and (b) a 36-item visual memory test on the basis of freeze 
frames from video material that was used in the stories as well as from "outtakes" (shots 
not included in the stimulus material). Subjects were 202 undergraduate communication 
majors who worked individually through the experiment. 

The results showed no statistically significant differences between factual memory scores 
in the high- and medium-correspondence conditions (the scores were 56% and 54% 
respectively). In the no-correspondence condition, factual scores were much lower (35%), 
but the auditory probe reaction times were not significantly different from these times in 
the other conditions. "This suggests that subjects were paying attention to the auditory 
channel, but that they did not remember much of the factual information contained within" 
(p. 23). Grimes discusses that this is consistent with the earlier mentioned observations of 
Colavita (1974) and Posner, et al. (1976) and also of Cohen (1973) who described that 
visual signals appear to go through fewer extrapolations than auditory signals before they 
result in memory codes. " Visual messages do not need as much attention to be efficiently 

encoded, and therefore get priority entry into the human information-processing 

system'" (Grimes, I^QO. p 23). If unrelated auditory- and visual stimuli are competing for 
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simultaneous processing, visual encoding primacy then keeps the factual information from 
being properly processed. 

Visual recognition scores for the high-, medium-, and non-correspondence conditions were 
66%, 55% and 64% respectively. The high visual recognition scores for the non- 
correspondence condition can partly be explained on the basis of the visual encoding 
primacy, that also caused the low factual memory scores in case of extreme audio-video 
conflict. In this condition, reaction times to the visual probes were relatively short, 
suggesting that the visuals were processed only to the point of recognition. The difference 
between visual recognition scores for the high-, and medium-correspondence conditions is 
statistically significant. It shows that audio-video correspondence influences the total 
amount of information that can be processed mentally per unit of time. In the high- 
correspondence condition longer reaction times to tone probes were positively correlated to 
visual recognition scores (r = .387, r2 = .150, p < .025): 

"This suggests that as subjects paid more attention to the narration, they were able to process 
the video better, and so they produced better visual recognition scores. In other words, as 
subjects paid more attention to what was said, they used their remaining attentionai capacity to 
integrate what they saw with what they heard. This result suggests that when the two 
information channels are correspondent, the narration 'drives' the processing of the visuals." 
(Grimes, 1990, p. 24.) 

This is seemingly in contradiction to the view of Kozma (1991, p. 192), who tends to 
conclude that comprehension of video appears to be driven by the processing of visual 
information: 

"This is apparent from a study by Baggett (1984), who varied the temporal order of audio and 
visual information within a video presentation on the names and functions of pieces of an 
assembly kit. In this study, the narration was presented in synchrony or 7, 14, and 21 seconds 
ahead of or behind the visual presentation. College students performed best on immediate and 
7-day delayed tests of recall of the synchronous and '* -second, visual-then-audio presentations. 
The worst performance was by groups with the audio presented first. This suggests that, in a 
video presentation, the visual symbol system serves as the primary source of information and 
that the audio symbol system is used to elaborate it." 
The two studies combined, however, need not yield a conflicting perspective. It may very 
well be that learners take the first few moments after switching to a new shot to process 
the new visual image to a level of recognition that is sufficient to be open to subsequent 
elaboration which is then guided by narration. In that case, Kozma' s conclusion that 
comprehension of video appears to be driven by the processing of visual information, has 
at least partly to be rejected. 

The question of excessive information load due to lack of audio-video correspondence 
appeared also in a study by Wember (1976). Wember analysed the informational structure 
of filmed background reports about the difficult political situation in Northern Ireland. He 
found that the filmmakers edited the visual layer of these films in such a way that they are 
dynamically paced and have an attractive rhythm. This had, for instance, the consequence 
that static shots were held relatively short (mean projection time: 2 seconds), while at the 
other end of the scale shots with movement of the camera and movement in front of the 
camera at the same time ran uninterrupted relatively long (mean projection time: 5.9 
seconds). The content of the films did, however, often not motive this editing style. 
Despite the fact that the filmmakers denied this, it seems that their professional consent 
that "film is movement" played them tricks. 

This is an occasion to stipulate that the media profession is differentiated. Media 
professionals who are schooled to work in the entertainment industry have, for instance, a 
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quite different view of their field than media professionals who are educated to serve 
educational communication. The need for differentiated schooling is, however, still 
recognized by scanty measure. The cooperation between filmschools or schools of 
cummunication and departments of educational technology is still very limited. This is the 
more problematic, now that multimedia technologies open a market where the combined 
skills of both worlds are inevitably needed. 

Wember perceived that the main effect of the dynamic pacing is that it attracts attention 
and prevents viewers from switching to another channel when these films are broadcast on 
television. But he was also able to demonstrate that the superfluous "Augenkitzel" 
(tickling the eyes), as he called it, demanded so much from the visual processing capacity 
of the viewers, that the intended, mostly rather abstract messages in the audio channel 
were not understood or at least not stored into memory although the viewers reported that 
they thought that they did. He concluded that the gap between pictures and commentary 
in the films he analysed in many cases forced the viewers to divide their attention between 
competing channels, with a consequence that they failed to really take in anything 
substantial of the presented information. Wember's study confirms that lack of 
correspondence between the visual channel and the audio channel is detrimental for 
learning. Unlike the findings of Grimes, in Wember's case the recall of the pictures 
dropped to a very low level. 

Wember's results pertain to attention processes in the visual channel that hindered the 
appreciation of the information in the audio channel. Another mechanism also exists. 
Boeckmann et al. (1988) assert that picture continuity takes precedence over a change of 
subject in the commentary: 

"When there is a change of subject in the commentary, even if this should amount to no more 
than taking up a new aspect of the same subject (such a change as is indicated in a written Cext 
by a new paragraph) and there is no corresponding structural marker (e.g. a cut, a graphic, 
flashing) the continuity of the information carried by the picture conceals the jump from one 
idea to the next and leads to reduced recall of the point in the commentary. Take, for example, 
the commentary that moved from 'the optional school subject' to 'the compulsory subject', 
while throughout moving pictures of a school class were shown. The analysis of the poor recall 
revealed that the experimental subjects had largely failed to perceive the transition." 
(Boeckmann et a!., 1988, p. 112) 
Heuvelman (1989) carried out several experiments with respect to between-channel 
correspondence and also came to the conclusion that "the relationship between the audio 
track and the visuals is of paramount importance to information processing by viewers" (p. 
167). One of the factors he studied concerned the effects of studio presentation on camera 
in comparison to realistic or schematic visualizations of abstract subject matter. His 
results led him to conclude that: 

"With abstract subject matter it is very difficult to match pictures with sound when using 
realistic visualizations. In that case, studio presentation on camera will lead to better results. ... 
With an increasing level of abstraction of the text, realistic pictures can only show a limited 
number of aspects or elements of what a certain idea or concept fully means. At the same time, 
in the eyes of the beholder, realistic pictures can have many potentional meanings. Selection of 
the intended meaning, i.e. that intended by the maker of the programme, is not imperative. ... 
In schematic visualizations, a selection, or abstraction, from reality has already taken place: only 
elements of interest are represented In that case picture and sound will show a better match 
and. consequently, processing and subsequent retention of the information will be better." 
(Heuvelman, 1989. p 16? ) 
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The view of Heuvelman implies that lack of between-channel correspondence does not 
only result in less learning if the information processing capacity of the learners is 
overloaded, but also if it leaves room for learners to invest their mental effort to derive 
non-intended meaning. That the use of schematization may heip to solve this problem was 
also demonstrated by Wember (1976), when he used animation instead of live images to 
support abstract content that was presented through the audio channel. 
In all, enough support exists to believe that proper between-channel correspondence 
promotes learning and that the amount of between-channel correspondence affects the 
information load of audiovisual presentations. How to control between-channel 
correspondence is, however, a complex matter. Factors involved are: (a) the concretencss 
or "visualizationess" of subject matter; (b) the matter of enrichment versus overload; (c) 
avoidance of distracting information; (d) amount of cueing of auditory content elements by 
optical markers or changes of shots; and (e) the mutual tuning of audio and pictures in 
general, inclusive of the wording and timing of commentary relative to the pictures. For 
every new product, the designers have to make content-driven local decisions to optimize 
between-channel correspondence with respect to these factors. It goes beyond the scope of 
this review, to elaborate design procedures to cope with these factors. A few general 
elements are incorporated in Figure 4 that serves as a contusion of this section. 

Superimposed text 

A particular form of between-channel correspondence is formed by superimposed texts 
that support narration. Heuvelman (1989) reviewed studies of Coldevin (1975), Nasser 
and McEwen (1976) and Nugent (1982) that show that the presentation of verbal 
information through two channels (visual and audio) yields information gain compared to 
audio only, but that no difference was found relative to the presentation of verbal 
information through the visual channel only (text on screen). This latter form is, however, 
rather artificial when the use of television is concerned. Heuvelman (1989) himself 
carried out a few experiments to find out whether short superimposed texts that support 
narration help recall of presented information. His earlier mentioned conclusion that with 
abstract subject matter, studio presentation on camera will lead to better results than the 
use of realistic visualizations was extended with the conclusion that "adding redundant 
superimposed titles can also lead to better results, but is of less importance and should be 
applied with care, taking into account the information density of the 'background' picture" 
(p. 167). 

Van der Heijden (1987) carried out an experiment with three conditions: (a) audio 
supported by short texts on screen against a neutral (blue) background; (b) audio plus live- 
action images; (c) audio plus live-action images and supporting captions. In this last 
condition only main points were supported by individual words or short statements near 
the bottom of the screen. The audio track was the same in all conditions. The 
programme took about ten minutes to present and contained information about a certain 
social service. This subject was rather abstract, the reason why the images in the last two 
conditions were, in many cases, not a 100% substitute for the screen texts from the first 
condition (which supported the narrated text in a way that resembles the use of the 
overhead projecter to present information point by point). Subjects were 61 students from 
the last year of a lower technical school and 62 students from the last year of a lower 
level high school. Conditions were about equally distributed over school types and 
learners. A pretest-treatment-posttest design was used. All tests were multiple choice. 
The results showed that the second condition (audio plus live-action images) did worse 
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than the other two conditions (audio plus text only and audio with live-action images plus 
text). The reason for this might have been that the imperfect match of the images "with 
the narration appeared a burden to the information processing abilities of the subjects, 
yielding a less effective use of the two channels than in the case where text on screen was 
available. 

The work of Heuvelman and of Van der Heijden give reason to expect that superimposed 
texts that support narration may be beneficial for learning. 

Contusions with respect to information load 

Research is not very conclusive with respect to picture complexity. A differentiated view- 
emerges in which complexity seems desirable to invoke interest, indicated if a need for 
concrete (real-life) referents exist, and to be avoided if the presentation of structures, 
relations, and principles is the central focus of the intended communication. These 
conclusions are summarized in Figure 4. 

The effects of using two channels and superimposed text appear to be dependent of the 
communicative intention which controls the kind of visuals used, and the relative 
importance of (oral) text. A few general conclusions are summarized in Figure 4. 

Summary 

The video components of multimedia applications may serve many different functions, 
such as providing content-motivated moving images, supporting mental imagery by 
visualizations, and stimulating personal (emotional) involvement with objects of study. 
The role of video will further be determined by the way in which video is positioned in 
relation to the non-video components of the system. In that respect, video may be chosen 
as the primary form of presentation around which all the other components are organized. 
But the use of video components may also be limited to support text-based materials 
where the use of video seems inevitable. Or as just one of the resources that users may 
encounter when browsing through a multimedia database. In all these cases, the video 
components should be designed with suffcient clarity to fulfill their roles in the 
communication process. In this contribution, research was reviewed with respect to 
production variables that should be properly controlled to eccomplish video segments of 
sufficient impact. In sum, the results are as follows: 

Camera fac'ors 

Research with respect to camera factors illustrates that the function of shot type is 
dependent on the circumstances under which it is used. The camera treatment has thus to 
be regarded as a flexible component. 

Research-based rules with respect to camera factors are very limited. A firm rule from 
practice is that for instruction the camera treatment should be content driven. Each 
camera angle ought to be motivated by its function in the presentation. 
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Source of information load 


Positive indications 


Negative indications 


Complexity through pictorial 
realism 


May promote recall and 
comprehension (Leahy, 
1983); invokes interest 
(Fleming & Levie, 1978) 


To many cues may interfere 
with learning (Parkhurst 
1975) 


Reduced complexity through 
schematization 


Simplified material is more 
effective for student 
achievement (Parkhurst, 
1975); schematic 
visualizations yield better 
learning results (Dwyer, 
1972) 


Simplified visualizations may 
lack meaningfulness (Leahy. 
1983) 


Combined use of the visual 
and the audio channel 


The combination can be 
more effective than the use 
of only one of the two 
channels (for instance: 
Kozma, 1991), specifically if 
the capabilities of both 
channels are property 
integrated (Dwyer, 1978); 
when the two information 
channel are correspondent, 
narration 'drives' the 
processing of visuals 
(Grimes, 1990) 


The combination can lead to 
excessive information load, 
which is detrimental for 
learning (for instance: 
Wember, 1976) 


Attention-demandig visuals, 
while the audio signal carries 
the main message 


Viewers are capable of 
concentrating on the audio 
channel and are able to 
learn from it, if monitoring 
the visuals does not cause 
cognitive overload (Grimes, 
1990) 


Visuals get priorty over audio 
messages dunng perceptual 
processing. In cases of too 
many information, the audio 
information may not arrive 
properly in memory (Grimes, 
1990) 


Reduced load of two-channel 
messages by 
schematization 


Schematizing abstract 
subject matter supports 
understanding and learning 
of narration-driven 
presentations (Wember, 
1976; Heuvelman, 1989) 




Lack of visual cues to mark 
transistions from one subject 
to another 




Visual continuation of a 
scene may mask substantial 
changes in the narration 
leading to less effective 
learning (Boeckmann et a!., 
1988) 


Focussing effect of 
superimposed text 


Superimposed text may 
support learning 
(Heuvelman, 1989; Van der 
Heijden, 1987) 


The use of superimposed 
text is of less importance 
than the prevention of 
distracting realistic visual 
images (Heuvelman, 198?) 
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Narration 

Video commentary should take the audience seriously, be presented in a clear, concise, 
and unambigious way, and talk directly to the audience if the subject allows it. 

Sound effects and music 
Findings from research with respect to the use of sound effects and music is very scarce. 
Here too, the main conclusion has to be that the treatment of these variables has to lean 
on rules from practice. 

Optical effects and special effects 
Findings from research with respect to the use of optical effects or special effects seems to 
show that deliberate content-related use of these effects may augment the quality of a 
message. 

Pacing 

Clear research-based rules about how to proceed when deciding on cutting rate and 
narration-related pacing appear not to exist. As with the other audiovisual design 
variables, in this case the design decisions are also subjected to professional judgement in 
each specific case. 
Technical qualities 

Picture quality has to be such that the images can be appreciated without technical 
imperfections. These requirement can be met by using professional video formats and 
professional camera treatment and editing facilities to produce video materials. 

Correspondance of video images and audio narration 
Enough support exists to believe that proper between-channel correspondence promotes 
learning and that the amount of between-channel correspondence affects the information 
load of audiovisu&j presentations. How to control between-channel correspondence is, 
however, a complex matter. Factors involved are: (a) the concieteness or 
"visualizationess" of subject matter; (b) the matter of enrichment versus overload; (c) 
avoidance of distracting information; (d) amount of cueing of auditory content elements by 
optical markers or changes of shots; and (e) the mutual tuning of audio and pictures in 
general, inclusive of the wording and timing of commentary relative to the pictures. In all 
cases, mental overload should be avoided. If the amount of information becomes more 
than the learner can handle, visual primacy seems to inhibit learning from the audio 
channel. 

Superimposed text 

There is reason to expect that superimposed texts that support narration is beneficial for 
learning. 

Despite the research findings and due to the limited direct applicability of these findings, a 
conclusion is that the judgement of an experienced producer and other experienced media 
personnel is indispensible in meeting professional standards. This puts media 
professionals in an important position because of the fact that media design directs the 
impact of instructional messages. The control over media variables is essential to 
optimize instructional communication. This factor counts for economic advantages of 
media, for the motivational effects of media and for media-specific didactics that extend 
the options for organizing instructional strategies in the sense of Reigeluth (1983, p. 18). 
ll is hoped thai the research reported here as well as subsequent research on (interactive) 
message design will lead to more systemi2ed knowledge to support design processes. 
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Designing Multimedia Applications 
Maud van den Meiracker 



Abstract 



The design and development process of multi media applications has to follow a structured 
approach. After a needs analysis, an educational design has to be developed. On the basis of 
this a functional design will describe the software architecture, which will be implemented in 
the next phase. In multi media applications content design is most important. Therefore high 
level editors are necessary. Also new expertise is needed in multi media design. One needs 
graphical designers and audio-video producers who are able to cooperate with educational 
teclinologists. 

Designing Multimedia Applications 

Introduction 

Computer based training applications have moved from character-based interfaces towards 
graphical user interfaces and from using only one data type, namely text, towards the 
integration of different data types, such as text, graphics, audio and recently also stills and (full 
motion) video. 

This shift from character-based towards multimedia applications has had implications for the 
design process. These implications will be discussed in this paper. 

The Design Process 

Over the years Andersen Consulting - ECC has developed its own educational software 
engineering methodology. This methodology contains a number of distinct phases. In each of 
the phases specific expertise is used to facilitate the process and to guarantee the quality of the 
final product. 

The main design phases are: 

- Needs analysis 

* Educational Design 

- Functional Design 

- Technical Implementation 

- Implementation and evaluation 

The main responsibility during the first two phases lies with the educational technologist. A 
computer scientist develops the functional and technical design and monitors the programmer. 
Implementation specialists, often people with an education degree, are responsible for the 
introduction and use of the developed computer application. 

For each of the 5 phases the implications for the use of multimedia will be discussed. 
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Needs Analysis 



The main questions which need to be addressed during this phase is which needs validate the 
use of a multimedia application. In order to answer this question, a short overview of types of 
multimedia based applications will be presented. 

Multimedia applications 

The following categories can use multimedia effectively: 

- corporate training 

- sales and marketing 

- entertainment 

- education/training 

- business desktop uses 

- integrated performance support 

Corporate training 

Thest applications are especially used for the training of new employees or the retraining of 
existing employees when new products or procedures are introduced. Multimedia is used for 
different purposes such as: 

- 'personal 1 introduction of the CEO and or other key-persons of the organization 

- visualization of the company, plant, etc. 

- use of simulations of the 'real' work which needs to be carried out 

- enhance motivation (training may be first introduction with new company) 

Sales and marketing 

The objective of sales and marketing multimedia products is to promote, sell and provide 
customer service for specific products and services. 

Examples are electronic brochures at a travel agent to select a holiday destination; at banks to 
choose between different kind of saving options; touch screen information booths at the 
Olympic games and other events, etc. 

Multimedia is being used to make an attractive presentation to the user who is used to the glitz 
and creative polish of the TV and movie commercials. 

Entertainment 

The expectation is that multimedia entertainment will grow rapidly with the introduction of the 
CD-I players, which are connected to a television. The home-market is the primary target 
group for multimedia entertainment applications. Applications consist mainly of interactive 
games. Multimedia is used to make the games more attractive by providing graphics, 
animations (cartoons), stills and video and of course sound-effects. The entertainment 
multimedia applications can be seen as an extension to the popular computer games like 
Nintendo and SEGA. 

Education/training 

It is expected that the market for multimedia education and training will quickly expand. With 
the prices of multimedia computers falling and the hardware becoming more powerful, 
multimedia will (partly) replace existing CBT and add many new applications which are highly 
attractive and effective The educational goal of multimedia application if to impart knowledge 
in the most effective way This can be realized for example in classes where students have 
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access to information from different cultures, languages, music, etc. Multimedia may provide 
more realistic and complete information and when integrated with well-designed computer- 
based training may enhance the learning process. 

Multimedia applications can be used very effectively in training situations where simulations 
are required. American Airlines, for example, uses interactive multimedia to cut down on the 
use of expensive full-scale flight simulators. 

Business desktop uses 

Multimedia usage for business desktop applications is not wide-spread but will certainly grow 
in the near future. The most promising applications are video mail and teleconferencing. 

Integrated Performance Support (TPS) 

Integrated performance support is mentioned separately because Andersen Consulting has a 
special interest in this area. The use of multimedia within integrated performance support often 
will be a subset of combination from the above mentioned categories. The goal of IPS is to 
support the knowledge worker. This can be realized with different IPS 'services' including 
CBT, advisory, on-line help, computer-based referencing and work flow. Multimedia can be 
very effective in the CBT, advisory and computer-based referencing services. 

During the needs analysis phase a good 'match' should be established between the specific 
needs for a computer application and the best solution. Of course, this is not new. Only now 
the option for multimedia is quite realistic, the educational designer needs to be familiar with 
the added value of multimedia and needs to know when the possibility of multimedia should be 
further investigated. Also the educational technologist should have some idea about the costs 
for developing a multimedia application. 

Even if multimedia seems to be the best fit for the specific needs, this does not necessarily 
mean that multimedia will be selected. Often budgetary constraints for both the development 
as well as the purchase of the delivery platform may lead to a different choice 

For the remaining part of this paper, we assume that the choice for a multimedia application 
has been made and we will discuss the implications for the different design phases following 
the needs analysis phase. 



'Educational* Design 

The 'educational' design is an important phase in the design process. Educational is placed 
between quotes since not all categories suitable for multimedia mentioned in the previous 
chapter, fall in the category education. 

This immediately points to one of the new skills for the educational designer within AC-ECC 
which has surfaced with the introduction of multimedia development. The focus is not only on 
an educational context but also on entertainment, point of sales which may include marketing 
aspects, integrated performance support, etc. Where training and learning were the most 
important goals for CBT applications, the different multimedia applications have different 
goals. The well-known and proven instructional strategies for CBT are not always applicable 
to other multimedia applications. 
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The educational design is of course closely linked to the outcomes of the needs analysis. The 
high level goals and objectives for the application to be d*.veloped have been defined in the 
needs analysis phase. During the educational design phase these goals and objectives will be 
translated into: 

• the overall structure of the application 

- the high-level navigation 

- level of user-control 

- instructional strategies and transactions or in case it does not concern a CBT-program the 
presentation modes 

- content specification 

Assuming that the decision has been made to develop a multimedia application, the following 
issues, specifically linked to multimedia, need to be addressed in this phase: 

Structure 

Where, when and how will the different data types be used in the application? 
Decisions will more often be based on availability of existing material (e.g. video and 
photographs); budget and time lines than on real design issues such as what is the most 
effective way to present information. 

High level navigation and user control 

How will the user access information and find his way throughout the application? 
Decisions have to be made with respect to the menu structure; use of hypertext; free browsing; 
free or fixed entry points, etc. This is especially important when using linear presentations such 
as video and audio. The user will need to stop video and audio at any point, backtrack at any 
point in time and sometimes the user should be able to skip certain presentations. 

Instructional strategies/ transactions 

The availability of more data types does not necessarily result in a high quality application. On 
the contrary, more can go wrong when using different data types. The correct choice for a 
certain data type in relation to the (instructional) goal is not always simple and to be honest we 
do not even know for certain the best match. 

If, for example, the goal is to introduce a new employee to the board of directors, just 
presenting a list of names may be less effective than presenting a list of pictures. However, it is 
still unclear if a short video presentation of each of the directors will be more effective than 
showing the photographs with some additional textual explanation. This will very much 
depend on the quality of the video as well as on the target population. 

Content specification 

The design should include specifications of all the content being used. Since more data types 
are being used, this means additional specifications in comparison with character-based 
applications where a description of font, letter size and style might have been sufficient. 
The actual scripting will take place in the next phases), but in this phase should be specified 
which data types will be used for which purpose. Furthermore, a description should be 
provided such as 'short video fragment showing the outside and the inside of World 
Headquarters'. 
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Functional Design 



The choice for using multimedia in an application has also implications for the functional 
design in which the overall architecture of the application as well as the detailed functionality 
will be specified. 

Andersen Consulting - ECC has successfully used the building block approach, of which the 
separation of (CBT) code and data, the development of both a (CBT) runtime program as well 
an authoring facility and the re-use of software building blocks are the main characteristics. 
Using this approach has resulted in very different design documents compared to the often 
used story-boards in multimedia design. The story-boards most often are linearly organized, 
describing one screen after the other. The AC-ECC approach results in a description of 
drivers, containing the way data is presented to the user on the screen with all the interactivity 
involved. The data are stored in a database which has to be designed by the computer scientist. 
In order for non-programmers to fill the database, an authoring program will be developed. 
This means that the functional design contains a description of the authoring facilities as well. 

No clear protocols have been developed within AC-ECC for describing the use of video, stills, 
sound and graphics within the functional design document. This is often done in a separate 
document, called detailed design or scripting document. 

Detailed design 

The detailed 'content' design describes in detail the different data types which are being used. 
This includes the following aspects for each of the different data types: 

Text: 

- font(s) 

- letter size(s) 
style 

- size of paragraphs (often linked to the size of the 'text-boxes') 

- use of horizontal and/or vertical scroll bars 
color 

Graphics: 

- description of pictures needed 

- size of graphics (often linked to the size of the 'graphics-boxes') 

- format (PCX, BMP, etc.) 

- metafile 
color 

- depends on screen (monochrome; VGS; super- VGA) 

- use of specific palette 

- use of corporate colors 
co-ordinates (where to place on screen) 

- transparency 
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Audio: 

- quality (11 or 22 MHz; 8 or 16 bits) 

- voice (male; female; child) 

- length of audio-fragments 

- with/without background music 

- specific effects 

- specific intonation 

Stills: 

- quality 

- size (often linked to the size of 'stills-box 1 ) 

- description of stills needed 

Video 

- description of video needed 

- quality (PLV or less) 

- length of video fragments (closely linked to costs and storage space) 

- size (full-screen or less) 

- combination with music or voice-over 

User Interface Design 

Another important part of the functional design is the user interface design. User interface 
design is important for all computer applications. However, because of the differences in 
multimedia applications as mentioned in the first chapter, it should be clear that some 
applications will be used by inexperienced and also infrequent users (such as point of sales and 
point of information). This requires special attention with respect to the user interface. 

The use of different data types may also result in rather chaotic and inconsistent applications. 
We often see that the philosophy "the more, the better 1 has been applied, which of course is 
hardly ever true. We recommend a rather 'dull' background using very quiet colors (like light 
grey) and predefined screen areas for video and still presentation and text and/or graphic 
presentation. The use of photographs, video and graphics will most likely bring enough color 
to the application. Therefor, we recommend to use only a couple of colors which have a 
certain meaning (for example red to indicate an answer is incorrect, green to indicate an 
answer is correct). 

Using different input mechanisms next to the mouse and the keyboard should definitely be 
considered. Touch screens, track balls and joy sticks may be much more user friendly for 
certain applications, especially if relatively little input is required. 

Overall, prototyping is a nice way to gradually design the definite user interface. This may 
cause some problems because of lack of in-house facilities and multimedia content material. 
But even without specific content, the user interface may be tested. A big advantage of AC- 
ECC's software engineering approach is that it is rather simple to change the user interface. 
Because of the separation of code and data, and the driver principle, changes in the user 
interface do not have to be realized all over the application, which is often the case when using 
authoring programs. 



Technical Implementation 

The technical design and programming phase for a multimedia application does not differ very 
much from designing character-based applications with the exception if different technologies 
and dedicated tools are being used, such as in the development of CD-I. 
AC-ECC uses Visual Basic 3.0 in combination with DVI-technology to develop multimedia 
applications. Also, when video is not a very important aspect of the application Video for 
Windows can be used, with the advantage that no special hardware needs to be purchased. 

Because of the basic building block approach, parts of the technical design and sometimes 
even code is being re-used in new projects. This is of course very efficient. 

Because of the large amount of data often used in multimedia applications, special attention 
needs to be paid to a DBMS. AC-ECC has used several applications such as DB-Vista, Access 
and Visual Basic's database engine. 

A totally new activity in this phase of the design process, specifically related to multimedia, is 
the data production. The multimedia data need to be produced and be placed in the database 
before the authoring process, actually linking all the data together within the program shell, 
can start 

Multimedia data production 

Professional production of multimedia data requires a multidisciplinary team. Each of the 
different data types requires different expertise. All these qualities will very seldom be 
combined within one person. 

Text productio n 

Depending on the type of application, different demands will be placed on the text quality. A 
good point of sales, for example, requires text writers with a marketing background. A 
training application requires a clear description of the subject matter and well-stated questions 
and feedback. Subject matter experts as well as educational technologists should work 
together to accomplish a quality product. 

Graphics production 

To make a multimedia computer application compatible with eveiything we see daily on TV 
and in magazines, it is imperative to involve a computer graphics expert for the graphics 
production. It is important to realize that a graphical artist who is used to paper design, is not 
necessarily suited to develop computer graphics. Best results are obtained when the graphical 
expert draws straight on the screen since drawing on paper and afterwards scanning of the 
image, often reduces the quality. 

The graphical designer should be creative and should also be able to listen well to the subject 
matter expert and to translate ideas into graphics, with all the restrictions the computer screen 
imposes. 

The graphics will have to be produced in the correct format (PCX, BMP or something else, 
depending on the technology used.) Also special attention should be paid to the filenames 
being used. Since all graphics will be stored in the database, it is helpful to think about 
meaningful names in order to facilitate the filling of the program shell later on during the 
authoring process. Actually, this applies to all data types used. 
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Next to preparing the text which will have to be spoken it is also important to select the right 

voice. Professional Voices' can be selected from a number of voice-bureaus in the Netherlands. 

Depending on a global description of your wishes, you can listen to audio fragments of 

different voices and select the appropriate voice for the multimedia application. 

Once the voice is selected and the text is completed, it is important to mark the text for 

specific intonation, pauses, etc. Very often the spoken text is used in combination with the 

presentation of stills and/or graphics. Timing will be done afterwards in most cases, but 

specific pauses should be taken into consideration when recording the text. 

In order to guarantee good quality, the recording should be done in a sound proof studio with 

good recording equipment. The text should be sent to the narrator beforehand for practice and 

should be recorded without any errors. 

Especially when multiple languages are being used, it is important to compare the lengths of 
the texts in the different languages. Rewriting may be necessary to match the lengths as well as 
possible. 

When the recording is finished, the anolog audiofiles will have to be converted to digital 
audiofiles. Here, it is important to choose the quality level. Highest quality will be obtained 
using 22 MHz and 16 bits. This of course will produce larger files than when 1 1 MHz and 8 
bits files are used. 

Digitizing can be easily done by using special tools such as Splice. 
Video production 

Special experts are needed to guarantee quality video in your application. You do not only 
need a good camera, sound and light technician, but also a good scenario expert and possibly a 
professional script writer. Producing video for a computer application requires different skills 
than producing video for a TV-show or movie. Oftentimes the video-clips in computer 
applications are rather short. This is because 1 minute video takes up 10 MB of disk-memory 
and longer video-fragments may reduce interest in the user. 

Decisions will have to be made with respect to using existing material or shooting new 
material. For new production it may be necessary to select actors, a studio, stage material, 
background music, narrator, etc. 

Just as with audio, video needs to be digitized as well. To obtain PLV quality, the video needs 
to be sent to special studios (e.g. Paris) where digitizing can be done on special high speed and 
volume computers for a considerable amount of money ($500 for 1 minute of video). 



Multimedia Authoring 

Another part of this design phase is the authoring process. As mentioned before, AC-ECC 
uses a software engineering approach where the code and the data are completely separated. 
This means that when the program shell has been coded by the programmer and the data have 
been produced and stored in the multimedia database, all the links will have to be put together 
in the authoring process. 

The authoring facilities are built by the programmer and are easy to use. Establishing all the 

links when using the authoring facilities does not require any programming skills. 

The author is often the educational technologist, but it can also be a subject matter expert. 



98 

O Q 



Testing 

Like with any software product, testing is an important part of the technical implementation 
phase. The difficulty with testing multimedia applications is the need for the delivery platform, 
which often is a 486 computer with DVT board and very large hard-disk. Sometimes, no 
additional computer is available next to the development computer. 

Because of the size of the application, you cannot just send a couple of diskettes to the subject 
matter expert or client to have the program tested. Installation of the application often requires 
an external hard-disk or a tape-streamer. Compatibility may be a problem, so it is important to 
have clear procedures. 

Because of the separation of the code and the data, it is advised to develop a testing protocol 
addressing the two parts separately. Graphics, audio and especially video will already have 
been tested before being digitized. This reduces the chances of having to make major changes 
to these parts of the data. 



Implementation and evaluation 

Developing a multimedia application does not guarantee proper and effective use by the target 
audience. It is important to support the implementation process and also to evaluate the use of 
the application. 

Just as with testing, the deliveiy platform for multimedia is somewhat more complex than with 
regular computer applications. The multimedia applications need to be properly installed. This 
is possible by using an external hard-disk, a CD-ROM or a tape-streamer. This almost always 
requires assistance from a technical assistant. 

Manuals are poorly read and not very much used. Furthermore, manuals are not suitable for 
point of sales and/or point of information applications and entertainment, since users should be 
able to access the application without any additional instruction (of course some instruction 
may be placed on the screen, such as: Select your language choice). 
For multimedia training applications and simulations, where learning goals are important, 
guidance and possibly a help desk should be considered. In the next two decades we may still 
expect many inexperienced computer users in the target audience of the multimedia 
applications to be developed. 

Evaluation of multimedia applications (and regular applications) often does not get very much 
attention. This is a pity, since much is to be learned from observing computer users and from 
discussing their experiences. Too often designers assume ways users will react and response, 
without ever having tested or validated this assumed behavior. Evaluation results are also 
strong selling points for the initial high investments which have to be made when starting a 
multimedia production. 
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Conclusion 



The design of multimedia computer applications has a number of consequences which do not 
apply for regular software engineering. 

- Content plays a much more important role than in traditional applications. This means that 
during the educational design phase decisions need to be made with respect to the added 
value of using the different data types in relation to the specific (business/ educational) 
needs. The (educational) designer will have to be familiar with the different possibilities 
and the added value of multimedia. 

- The interactivity of multimedia is not very well known. We should be careful that 
multimedia programs do not become video or slide-shows. 

- Multimedia development definitely requires multi-disciplinary teams. This poses an 
additional burden on the project management of a multimedia production. Managing the 
creative processes of graphical designers, script writers and video producers should not be 
underestimated. 

Timelines will be under much more pressure, since there is much more interdependency in 
multidisciplinary teams. 

- Data production will seldomly be done in-house, because expensive equipment and studio 
facilities are required. This creates additional dependencies. 

- 'Standardized 1 protocols will enhance the communication between team-members. 

- In order to develop maintainable applications, it is important to keep the program code 
and content data separate. This is realized within AC-ECC by using a regular software 
engineering methodology for multimedia productions. Many of-the-shelf multimedia 
authoring tools do not support this separation of code and data, which reduces the 
flexibility very much. 

- Cost estimates of multimedia applications are still not very easy to make. Much depends 
on the preparation of the data and the desired quality of the data. 

- With the prices of hardware reducing, multimedia applications will take a growing market 
share, which will only stimulate the demand for more multimedia applications since critical 
customers will not accept anything less attractive. 
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Multimedia development tools 
Jan Schoenmaker, Erik Nienhuis 



Abstract 

When specifying multi media tools one starts with defining the optimum between 
the needed efficiency of development and the flexibility for an author. Given the 
importance of content design in multi media applications, high level structure and 
content editing tools should be available with which an author can design and 
develop applications in an easy and cheap way. As an example a dedicated 
authoring environment for multi media development is being described. 

Criteria for tools selection 

Efficiency and flexibility of tools 

The discussion on the functionality of tools for developing multimedia applications is partly 
the same as in the case of regular software development. Does one need high flexibility than a 
regular programming environment will certainly support the development. However the 
efficiency of the development process may be quite low. In order to increase this efficiency 
one builds environments in which a number of facilities support faster development. Where 
does this increase in efficiency come from? This is due to the definition and implementation on 
beforehand of certain functionalities which may be useful in an application. The developer has 
only to put in some parameters and part of the application will be generated automatically. In 
regular automation we then talk about fourth generation environments, in training about 
authoring systems. However, one has paid a price in using such a tool: because part of the 
functionality is pre-defined, one has less flexibility. One can say that it is dependent on the type 
of application and context factors like maintenance but also expertise which is necessary which 
will decide upon the selection of tools. 

Expertise as a third criteria for selecting tools 

As said, expertise of the developer is a major variable in using a tool. Although regular 
programming environments are very handy, programming is a highly specialised activity. 
This is a major issue in application domains like training, where one primarily needs expertise 
in instructional design, human computer interaction and learning psychology. 
Computer based training therefore not only demands for software engineering knowledge and 
skills but also content design skills. 

Here comes the issue with multimedia. In the past content design was restricted to writing 
dialogues and using simple graphics. Now, with multimedia the demand is for 'scripting* 
content in which as well the cognitive, interaction oriented activities with computer based 
training as the more affective, contextual activities of film are being combined. So, a designer 
has to be as well an instructional designer with a strong focus on cognitive aspects as a film 
maker with a focus on contextual and affective aspects. 

So content design in multimedia computer based training has become more complex. 
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Developing a new type of tool with new technology 

One can distinguish three criteria when selecting or defining authoring tools: efficiency, 
flexibility, expertise. One can put the well known tools like authoring systems, authoring 
languages and programming languages in relative position. Authoring systems have a 
restricted functionality and flexibility, but high efficiency or power of development. With 
programming environments it is the other way around. 

Given nowadays technology one can take a new perspective however. Available are now 
generic tools like user interface management and data base tools. Making using of this 
technology has brought the opportunity to develop a new kind of authoring tools, leading to 
as well high efficiency and flexibility in specifying all kinds of functions. 
With this technology an application can be seen as consisting of three distinct parts: the user 
interface part, the functional part and the database part. Certain environments combine a user 
interface management system and regular programming features, like for instance Visual 
Basic. 

Others combine this also with a database part, like it is in hypermedia systems. However, the 
storage of data in a hypermedia system is based on the card concept. A card represents the 
userinterface part with a fixed link to a content part. Linking cards to each other means fixed 
links, leading to maintenance problems if one wants to change links. Using regular database 
technology , implies that maintenance can be done in an efficient way. Therefore, in general we 
prefer the use of regular database technology to organize the different datatypes in multi 
media. 

Focus on content design 

What does the availability of these new tools mean? 

These hypermedia systems and programming environments are in itself already high level 
tools. They have on board a user interface management system and also support a number of 
relevant functions in developing computer based training: user interface and storage of 
(multimedia)contents. They lack functionalities typical for authoring systems like pattern 
matching for processing of answers or registration facilities. But given the high level of 
sophistication of these programming environments one can built these relatively easy. 
Therefore we come to the statement, that making use of regular software engineering 
architectures and tools, one can built environments which are competitive with authoring 
systems in terms of efficiency. These environments will be built on the basis of specific client 
needs. So the flexibility issue is resolved on beforehand. 

Maybe that the most important characteristic is that one builds these tools in such a way that 
authors just have to use structure and content editors. Because the functionality of the 
dedicated authoring tool is defined on beforehand, the author can concentrate on course 
structure and course content. As stated before this is highly critical in multimedia design 
because of its complex relations between text, graphics, audio and video. 

In the next chapter an overview will be given of the design considerations of this method and 
of a specific authoring environment developed by Andersen Consulting - ECC. Because it is 
based on defining and implementing basic building blocks which can be combined as bricks in 
building a house, this method is called the basic building block approach. 
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The basic building block approach. 



The ECC building block approach has four main aspects: 

• separation of CBT code and data; 

• development of two programs in a project: a CBT runtime program 
and an authoring facility; 

• organizing software into instructional units; 

• customizing reusable software building blocks for the specific needs 
of a project. 

U has to be noted that each of these aspects is an integral part of "the" ECC building block 
approach. The approach is not just about instructional units, nor just about re-usable code. It 
is about all four aspects combined together. 

Below, each of these aspects is further explained: 

Separation of CBT code and data 

Separation of code and data is a fundamental principle of software engineering. It means that 
the functions or logic of a program are separated from the data on which these functions 
operate. However, for developing CBT, this has not always been common practice. 

In the ECC building block approach, the CBT logic is rigorously separated from the course- 
specific CBT data. With course-specific CBT data, we do not only mean the course content, 
but also the course definition. The course contents are the texts, graphics, audio, stills or 
video that contains the information to be learned by the learner. The course definition specifies 
how this information is organized in the course, e.g. in terms of modules, the exercises 
contained in a module, the sequence of presentations, etc.. In a way, the course content is the 
training material, the course definition is the training structure. 

The CBT logic is the way in which this data is presented to the learners on the screen, with all 
the interactivity involved. In the ECC building block approach, this logic is programmed in the 
form of so-called drivers, A driver is a piece of code that takes a (corresponding) set of CBT 
data as input, in order to deliver this data on the screen according to the programmed logic. 
For example, a multiple choice driver takes data like the question-text (string), the answer 
texts (n strings), and the number of the right answer (integer) as input to deliver this multiple 
choice question on the screen. 

It is obvious that, once a driver is programmed, it is very cost-efficient to make a lot of CBT 
with it, for only the data has to be specified. In a way, the CBT data (including the course 
definition) acts as an extensive set of parameters for the CBT code. One can now 'author* CBT 
by simply parametrizing it. This approach is very different from CBT development with 
conventional 'authoring environments' (such as Syllabus or Authorware), where the code 
(interactivity, logic, branching) is completely interwoven with the course definition and 
contents, and where, consequently, authors and instructional designers are programming the 
CBT and editing the course at the same time. 
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Two applications: CBT and Author 

Based on the separation of code and data, the CBT will be delivered through a CBT program 
with the CBT logic, containing all the drivers, and a CBT database with the course data 
(contents and definitions). However, in order for authors to fill and edit the database, a second 
program is needed: an authoring program. In the ECC building block approach, the CBT 
program is usually called course shell, and the authoring program authoring shell The 
diagram below shows how programs and database relate to each other: 



This approach makes it possible for non-programming subject matter experts to directly author 
the course, without being bothered by programming details. An important advantage is further 
that the course, delivered through the CBT, becomes b^y maintainable, because at all times 
the authoring shell can be used to edit or extend the course. 

It has to be said that, sometimes, a third program is built for CMI (Computer-Managed 
Instruction) purposes: an instructor shell. This program could for instance communicate with 
the course shell, using datafiles, to enable an instructor or coach to view registration and 
results of learners, and to pre-select certain course modules for them. However, this is not 
really a integral part of the building block approach. 

Instructionally meaningful units 

The building block approach requires fixing instructional transactions into software 
components, in a way which is specific for the requirements of a CBT project. Not only 
'screen 1 level, but also levels above that (invisible), like modules, series of exercises, test or 
practice use of an exercise driver, progress registration, etc. 

In the ECC Building Block approach, CBT is developed from an instructional design based on 
the application of a limited set of instructional units. These units are e.g. a type of 
presentation, a type of exercise (for instance multiple choice), but also higher level 
instructional units, like 'module', or 'menu 1 , or Vegistrate learner progress*. After this, in the 
functional design of the software, the instructional units are then translated into a number of 
units of 'drivers' (interface and code) combined with dataformats for the database (e.g. record 
structures). It has to be noted that in the software design, some drivers will be added to the 
'instructional ones', based on software architectural considerations (e.g. defining a driver which 
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controls the running of a related set of lower level drivers), or on functional considerations 
(e.g. drivers that control initialization of the environment etc.). 

Because the author shell uses the building blocks as well, the author can only enter data in the 
format which has been defined in the instructional transactions. 



Reusable library, but customize 

• re-using the existing library of building blocks, but still customize them for the specific 
instructional needs of a project. 

Over the past 2 years, a library of ready-to-use building blocks has been developed. These 
building blocks have been made generic, so that they are easily reusable for new projects, by 
strictly enforcing coding and documentation standards. However, because of this generic 
nature of the building blocks, they are quite abstract and complex. Consequently, re-use of the 
building blocks is best done by ECC designers and programmers. Re-use by non-ECC 
developers will prove difficult and is not recommended. 

Although the building blocks can be re-used directly, without any changes, it has been 
common to customize the building uocks, i.e. adapt them to the specific needs of a new 
project. The level of customization may be small, such as changing the place of a button, or a 
small change in the logic, or changes may be large, such as changing the user interface style, or 
changing the driver- and data structure. Because of the strong separation of interface, 
functionality and data, customization is relatively easy, and therefore always done. 

The customization usually results from a sort of negotiation process between the wishes of the 
instructional designers and the feasibility of the software implementation. This negotiation 
process is dene in the functional design phase, and is an important task of the functional 
designer. Thus, a good level of both customization and re-use is ensured. 
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The ECC multimedia toolkit 



The ECC mtiltimedia toolkit is an environment for developing advanced multimedia 
applications. It is a specific implementation where use has been made of the building block 
method. A choice has been made for a certain course structure, presentation functions and 
interaction functions. It is therefore a dedicated high level authoring environment. 

The ECC multimedia toolkit consists of three major components: 

• authoring shell: supporting the authoring process; 

• instructor shell: supporting the instructors role like allocation of courses and registration; 

• user shel! the application for the learner. 

Besides one needs: 

• text, graphics, audio, video editing tools: these are tools for the author with which 
contents are being produced. These are generally available tools on the market. 

Hardware and software requirements 

Hie ECC multimedia toolkit is available for three levels of multimedia: 

• Supporting text, graphics, stills, animations, audio, video: 

This is based on DVT (Digital Video Interactive) or Video for Windows. 

When using DVI, supporting full screen video one needs a DVI Action media board. 
With Video for Windows no additional hardware is needed, except a voice card. 

Furthermore the minimum hardware requirements are: 

a hard disk with at least 300 MB (1 minute of video takes about 10Mb, 1 minute of 
audio about 1 Mb in DVT mode) 
a mouse 

a 33 MHz personal computer with 80486 processor 
8 Mb internal memory 
a VGA colour monitor 
VGA-board with Feature-connector 

Amplifier (with normal line Input) and headphones or active speakerset 
Window version 3.1 

• Supporting text, graphics, animations, audio: 
The same configuration as for Video for Windows 

• Supporting text, graphics, animations: 

No voice card needed, less mass storage (i.e. 100 Mb), eventually 80386 computer with 
4Mb RAM. 



106 

1< 6 



Course structure 

The structure of multimedia courses which can be developed with the ECC Multimedia toolkit 
is pre-defined. The following picture presents the structure of a course. 
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As one sees in the previous picture a course made in ECC-Multimedia Toolkit contains one or 
more modules. These modules can be a Test, Video or Standard module. 
A Standard module consists of: 

Introduction 

Overview 

Practice 

Test 

Introduction: 

The introduction contains a text window with the objective of the module and (optional) a full 
screen introduction video clip. The objective text describes the goal of the module. 

Overview; 

This part of the module presents the main points of the module. These main points are 
presented in an audio fragment in combination with stills, graphics and text, a so-called 'audio 
page 1 . The objective of this overview is to give the participant a 'advanced organiser 1 for this 
module. In this part the content of the module is placed in a wider context and also the route 
trough the module is given by placing the different Topics on the screen. 
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Practice: 

In this part of the module the participant must answer questions and can get more information 
by selecting presentations. These presentations are made by using the multimedia database. 
Hints can be provided for selecting the right presentation. 



Issti 

The test consists of questions without feedback, 
the participant. 



After the test a study advice will be given to 



Multimedia database: 

Information will be presented in a 'information bank 1 organised around the topics in the 
overview. Clicking on a topic in the information retrieval can give direct presentation or a sub 
menu with 'sub menu items'. Each item contains a presentation. 

Presentations consists of audio pages, static pages or video pages and are presented in a 
window on the right-hand side of the screen. Video pages consist of an specific audio-video 
fragment. 

An audio page contains an audio fragment (mostly a narrator speaking) with a series of stills, 
graphics or text. A static page is atso an audio page with only one still, graphic or text. 

The other module types are Test and Video module. The Test module contains only questions 
and no feedback. After the test the score will be presented. A video module contains a video 
fragment to be presented after a target text in a window. 

Course definition 

After one has started the ECC multimedia toolkit the computer presents a screen, in which one 
defines the course. 

Here one can: 

• edit the title of the course; 

• define the language, in which the texts and buttons are being presented; 

• select the video fragment, which will be presented as the trainees start-up the course; 

• select a menu graphic, which will be presented after the opening video fragment. The 
module names are presented over the menu graphic; 

• edit the dictionary (book), which contains explanations of relevant words used in the 
course. During the course the trainees can look up these words. 



Creating and editing a module 
Each module may consist of: 

• introduction and overview; 

• presentation organised in topics and items; 

• standard tasks or exercises consisting of multiple choice, multiple select and matching 
assignments; 

• task tests consisting of multiple choice, multiple select and matching assignments. 





Introduction of a standard module 

The introd uction of a standard module is always presented to the trainees at the start of a 
module. 

An introduction may consist of: 

• target presentation; 

• video presentation (optional); 

• audio presentation (overview). 

The introduction starts presenting the target text, which describes the objectives of the module 
selected by the trainees. Subsequently the course may show a video, which is followed by an 
overview of the module. This overview consists of an audio presentation combined with topic 
names, stills, graphics or text. The function of the overview is an 'Advance Organizer 1 . 

An audio presentation can be combined with topics, stills, graphics or texts, which are shown 
in the list with scrollbars as filenames. 

Creating presentations 

The presentations are hierarchically organised. Each module has one or more topics, which 
may includes items. An item leads directly to a presentation. 

There are three types of presentations one can create: 

• vidvO presentations; 

• audio presentations; 

o static page presentations. 

Audio An audio fragment combines an audio fragment with a series of graphics, stills or texts, 
and can thus be compared with a 'slide show*. Such a fragment usually comprises of 
more than one page. 

Static This is a presentation of one page containing graphics, stills, text or a combination of 

these images. 
Video A video only shows sn entire video fragment. 

Editing presentations 

Audio page 

An audio page shows an audio fragment combined with one or more graphics, text or stills. 

\ , For each page one has to define: 

• the filename; 

• type of image (still, graphic or text); 

• timing (the time of appearance of the image in the audio presentation); 

• position on the screen (see below for type of lay-out); 

• a visual curtain effect at the transition of pages. 
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fcdh Audio page 




In this new window one can conduct the following actions: 

Direct presentation Show the transition of the current fragment to next one with or without 
a visual curtain-effect. 

Test Test the appearance of the previous and current page in the 

audiofragment. In the right window one seen the image and hear in the 
audio. 

During the test one an; able to select the appropriate timeperiod of 
appearance via the button Insert time'. 

Insert time Change the timeperiod of appearance of the current fragment. 

The start of this timeperiod is determined at the moment one press on 
the button Insert time 1 during the test. 

Play (>) Play the audio presentation. 

Pause (||) Stop the audio presentation. 
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Static page 

A static presentation is a presentation of a graphic, audio, a still image or text. In comparison 
with an audio presentation, there is only one page, which may have one or two images. 



Edit Staticpage 




Video page 

A video page is a presentation of running video (combined with audio). 
Create exercises (task) 

There are three instructional transactions available supporting interaction: 

• matching; 

• multiple choice; 

• multiple select; 

Matching 

A matching exercise consists of two columns, which both contain words. The task of the 
trainees is to match the words in the two columns,, which belong together. At the end the 
program checks the answer and offers feedback. 
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Multiple choice 

A multiple choice exercise presents to the trainees a question text and 2 to 4 answer options. 
The answers can be presented as text or stills. The task of the trainees is to answer the 
question by selecting one option. The program checks the answer and offers feedback. 

Multiple select 

A multiple select exercise has an instruction text, a question text and two feedback texts just 
as the multiple choice has. The difference is that multiple select may contain more than one 
good answer.After a response the program checks the answers and offers feedback. 

Create tests 

Tests make use of the same three interaction types, multiple choice, multiple select and 
matching. 



The instructor shell 

Different participants can be connected to a course by using the instructor shell. 

The instructor shell comprises of the following parts: authorisation, participants, install options 

and registration. 



Extending the authoring environment 

So far the multi media toolkit has been described with its specific functionality. What if one 
wants new functionality? One could specify and implement for instance new presentation 
forms, new interactions or new support functions and integrate them into the existing 
environment. Also editors have to be adapted then. However, if there are essential changes in 
the structure, one might build a new environment . Re-use will be made then of the lower level 
drivers. These drivers will often be customized to particular needs. Based on our experience 
one can say that re-use is often possible to 50-80%. 

This approach has proven to be optimal in terms of development costs and client demands. We 
then compare this approach on one hand with making use of tools having no flexibility and on 
the other hand with developing each application from scratch with a regular programming 
environment. 




Putting multimedia into practice 
Mark Vegting 



Abstract 

In this paper, a description of how multimedia can effectively be used in practice is presented. 
Two applications are discussed, which were both developed for a touroperator. One 
introduction course and one course on basic sales skills. In both courses a combination of 
instructionally relevant multimedia information retrieval and questioning was used. Evaluation 
shows that the goals were met and participants were satisfied with the results. 



The company 

SUN Intentional is a Tour operator, based in Belgium. The company operates in four 
different countries: Belgium, the United Kingdom, the Netherlands and France. The company 
started in the early fifties as a small one-man company, arranging transport for English 
travellers who came to Belgium. Later, they were transported from Belgium to the 
Mediterranean Sea and the Alps. Adding more and more services, the company grew larger. It 
began to organise the transport and activities itself. In the seventies, the company took over an 
important player in the Belgium travel industry. In the eighties many shares were sold, but the 
majority remains in the hands of the founder. Because of the developments in the travel 
industry and in the European Community, Sun International decided that it was important to 
become a real international firm. The capital that was drawn with the selling of the shares was 
invested in the take over of other companies, e.g. in the UK. The latest merger took place in 
1992, also in the UK. Sun International now offers a great variety in travel industry services. 
From bulk travels to specialised city trips for a great deal of travellers. This is realised through 
a conglomerate of several separate companies. All these companies are part of Sun 
International, but operate under their own name. The name Sun International is reserved for 
several general, centralised activities. 



The target population 

In the entire travel industry, and SUN International is no exception to this, there is a relatively 
high turnover of personnel. This means that there is a continuous flow of new employees, who 
must be introduced to the company, but who also must learn the reservation system and sales 
skills. 

There is a difference between the employees in the UK and on the continent. In the UK, the 
average level is secondary education. The target population can be typically summarised as 18 
year old girls. Although there are also some men and some older women in their fifties. On the 
continent, the level of education is higher: most of the workforce has attended some years of 
higher vocational education. Consequently, the average age is higher: around 20. 
In general, the employees only speak their own native language. 
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The need 



Due to the rapid growth in the eighties and nineties, some reorganisations were necessary. The 
newly acquired companies and internal changes, combined with the high turnover-rate, led to 
the situation where many employees did not have a good overview of the activities of the 
company. Also, did they lack a feeling of corporate identity. They also were unable to position 
their own activities within the activities of the company. 

Also, many employees were not sufficiently equipoed to perform their tasks 100% well. 
Several causes were due to this: The rapid growth of the company, different reorganisations, 
and the desire to implement new management styles. The latter cause meant (amongst others) 
empowerment of the employees. The tasks of the employees were expanded. For example: the 
separate 'complaints department' was integrated in the sales-function. But also the reservation 
system tliat was used was going to be changed. Although the intentions of the management 
were good, the skills of the employees to perform all these tasks had not been upgraded 
adequately. 

In general terms, the need for training can be summarised as follows: 

• Lacking the insight in Sun International's corporate identity; 

• Unclear organisation structure; 

• Insufficient knowledge of the activities of their own company; 

• Insufficient sales skills; 

• Insufficient product know»edge; 

• Turn over of new employees. 

The objectives 

The needs that were identified led to corresponding objectives. But Sun International also had 
additional objectives. For example, it wanted participants to regard Sun International as a 
company with a 'human face', a company that cares for its employees. To realise all this, two 
applications were developed: a 5 hours multimedia application to develop the attitude 
regarding the corporate identity, called SEC (Sun International Introduction Course), and an 8 
hours multiiredia-application focusing on the product knowledge, systems knowledge and 
sales-skills, called SISI (Sun International Sales Introduction). 
SEC was to develop the feeling of a corporate identity and to explain the organisational 
structure, as well as the benefits of being part of a large organisation. The application 
introduced participants also to all the companies activities. 

Both applications were created in two languages: English and Dutch. The French version will 
be created later. 



The rationale 

Why was multimedia chosen as a solution for the identified needs and this target population? 
The main reasons are the establishment of a corporate identity and showing a human face. A 
corporate identity is intangible. An important aspect of it however, has to do with the attitudes 
within a company. A positive attitude towards the company is inevitable of course. Situations 
in which attitudes have to be developed can very well be supported by using realistic images 
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and role modelling. Another reason was the motivational aspect of using audio and/or video. 
Above this, multi-sensory environments have a big impact on people, because it comes close to 
real world situations. This leads to easy association with own experiences and consequently 
better retention. Another goal of the application was to give the employees the feeling that the 
company cared for them. A multimedia environment, was a way to realise this. 
Other arguments can be found in the business the company is operating in. The travel industry 
is very much visually oriented. Nice pictures of holiday destinations serve as the best selling 
argument. The employees are confronted with this every day. The application had to be 
connected to this way of presenting information. Another practical reason was that the library 
of photographs with Sun International was very big. 

By making use of different media it was possible to make advantage of the medium that was 
most suitable for its purpose. Because running video (if at the right time) can have the highest 
impact on people, this was reserved for the most important elements of the application like the 
welcoming speeches and the presentations on the cornerstones of the company. Although 
pictures can tell more than a thousand words, it is in most cases very useful to have some 
additional explanation of the pictures. That is where narrators were used. If the narrator's 
information was really crucial for the participants, the story was supported by bullet points. All 
these media could be combined in one application on a PC. And alternation between the best 
suited medium led to an efficient, attractive application. 



The application 

Both SIIC and SISI are comprised of different modules. For SEC, there are some general 
modules, which are the same for all employees, and one company-specific module. The 
company-specific modules, contain elements which only relate to the acthdties of that specific 
company. For example, the countries or cities they focus on, the brochures they use, the people 
who work there, etc.. These company specific modules will only be taken by participants who 
work with that specific company. After logon, the participants will be directed to those parts of 
the contents that relate to their company. 

SIIC is comprised of eight modules: 

• Pre-test 

• Introduction 

• The travel Industry 

• The history of SUN 

• SUN as a group 

• Company-specific module 

• Test 

• You and SUN. 

A pre- and post-test are included to be able to assess an increase in knowledge with the 
participants in their current knowledge about the elements that are covered in the application. 
These tests are comprised of 20 multiple choice questions. The first 15 questions of each test 
cover the general modules, where the last 5 questions are on the company-specific module. 
The pre-test appears only the first time the participant starts the application. After the 
participant has finished it, the participant is confronted with a main menu. Here, the participant 
is free to choose any module. 
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"Introduction" is the first multimedia module. Introduction 1 is a video in which the founder and 
chairman of the board of directors welcomes the new employee. The three cornerstones on 
which Sun International is based are introduced as weil. This module is a linear video. 
The modules "Travel Industry", "History", "Sun as a group" and "Company specific 5 *, ail have 
the same structure. This structure is comprised of the following elements: 

• Short audio/video-clip to gain attention: Introduction; 

• Overview of the entire module, in which an advanced organiser is built; 

• Practice, consisting of questions and information retrieval: 

• Questions regarding the contents (4 types: Multiple choice, Multiple select, Matching, 
Multiple choice with photos); 

• Information retrieval system to retrieve multimedia presentations on certain topics, 
providing relevant information to answer the questions; 

• Module test. This test can only be taken after all questions have been aaswered. 
The last module "You and Sun" is a second "welcome to the company". Thfc time b> the 
managing director of the company. This speech focuses more on the future of the new 
employee with the company and stresses the core values. The main goal of both module? 
"Introduction" and "You and Sun", is to show the human face of the company, and to lealbc 
the biggest impact on the participants. 
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The structure 

Let's discuss the structure of the 4 modules in more detail. Look at figure 2. 
Standard module: 



Introduction 



Overview 



Practice: 



Questions 



information 
Retrieval 



Test 



Figure 2: Overview of the structure of a standard module. 
Introduction of a module 

By clicking on the button for a module, the objectives of that module appear. When the 
participant wants to reach those objectives, (s)he continues. A short video clip is then started. 
Each module has its specific clip, which relates to that module. The "Travel Industry"-module 
starts with a collection images. Images of players in the travel industry, of holiday travellers, of 
types of transportation, of types of accommodation. These elements are all covered in more 
detail in that module. The module "The history of Sun" starts with a black and white movie, 
showing how the reservations in the early fifties were made at Sun International. "Sun as a 
group" shows musicians playing separately and combined performance of the whole band. This 
is to symbolise the added value of operating as a group. The company specific module starts 
with a similar collection of images as the Travel Industry, but the images are more focused on 
the activities Sun performs. 



Overview/advance organiser 

After the introduction of a module has finished, an overview of the main topics of that module 
is presented. This presentation is comprised of graphics and pictures, with a narrator 
explaining. This explanation is supported by keywords, which appear at the moment the 
narrator mentions them. This way, an advance organiser of the content of the module is 
presented. 

Questions/tasks 

After the advance organiser is built, the first question appears. All questions allow the 
participant two tries. Should the correct answer not have been given, the correct answer will 
be presented. Only when the correct answer is given, either by the participant or by the 
application, the participant is allowed to continue with the next question. 
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Multimedia information retrieval 

Treating adult participants as adults who are responsible for their own learning and who are 
able make choices is one thing. But also for adults just browsing through a database and 
looking at presentations that appear interesting, is not the most efficient way to learn. 
Although it can be effective. It is more efficient if the participants are guided in some way at 
their exploration of the data. The questions/tasks provide this guidance in the following way: 
A question is asked about a certain subject. The participant is unable to answer this question, 
but he or she knows that there is more information on this subject available in the database. 
The search for information is more guided in this way; the participant has a focus. If the 
information is hard to find, or if the participant makes a mistake, the feedback will explain 
where to find trie correct answer. The participant has the bullet points of the advance organiser 
from the overview at his or her disposal. These bullet points can be selected, which leads to a 
menu. The items in this menu can be selected to get a presentation on that specific item. This 
can be a video, a series of graphics and photographs with a narrator, or text. Sometimes, the 
presentation appears directly after selecting a bullet point. 

So, the participant has complete freedom in the exploration of the multimedia database. 




Fig.3. Overview of the main working area oft standard module: On the right the window for 
questions appears, on the left the buttons for exploring the database are available. 
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Fig. 5. Example of static page-presentation: graphic with text, from the multimedia database. 
The window with the question remains available behind the window with the presentation. 
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The design and development process 

At first a needs analysis was conducted by the instructional designer. The current situation was 
reviewed, including the current training situation and the current activities of the entire 
company. The main goals that the company wanted to achieve were identified and the way the 
training could support this. Interviews were conducted with the management of the company, 
and with employees, both existing and relatively new ones. The conclusion from this analysis, 
were the needs as listed in The need'. 

The starting point for the design was that participants should be in control of the application. 
This was even more important because the participants are adults. The context (and the 
content) did not allow for simulations, but other options were possible. The focus of the 
application was on introducing new employees and establishing corporate identity. The content 
is therefore more in the affective than in the cognitive domain. This is not to imply that no 
cognitive activity is necessary, of course. Also, the module about the travel industry contains 
more facts, concepts and explanation than affective elements. But in general, this is true. 
A team with the instructional designer, the technical designer, and the client decided that a 
multimedia information retrieval system would be used. But parallel to this information 
retrieval system, which allows free exploration, guidance and focusing was necessary. This was 
realised through the use of questions. 

Visualisation 

By choosing this parallel approach, it was relatively simple for the technical designer, to design 
the technical architecture. Although the two functions were available parallel, it was not 
necessary to transfer parameters from one element to another, or from one module to another. 
The difficult part was with the detailed instructional design. It was necessary to design/script 
interesting, short, relevant and understandable presentations. To establish the corporate 
identity, the three cornerstones of Sun International had to be present in most presentations. 
The length of the presentations was kept to 60 seconds maximum. Experience had learned that 
longer presentations were tiring, and participants did not remember what was explained. A 
multi-disciplinary team, comprised of an instructional designer, a technical designer, a 
graphical designer and a video director designed the entire application. 
Three types of presentation were designed, called Video page', 'audio page' and 'static page'. 
Video pages are digitised videos and audio. 

Audio pages are explanatory/interesting/ motivating graphics and pictures, supported by 
explanations by a narrator. The appearance of the pictures is timed with the digitised audio of 
the narrator. 

Static pages are graphics or pictures with supporting written text instead of audio, or just a 
piece of text. 

Authoring 

Guidelines and templates for scripting were developed. This allowed employees from Sun 
International to script presentations as well. Also were they able to create questions by using a 
dedicated authoring system. Of course, it remained necessary for the instructional designers to 
review the presentations and questions, but a good mix could be established between necessary 
content expertise and instructional quality. 

Because of this approach, and the technical architecture, it was possible to program the 
software parallel to the scripting and realisation of the presentations and questions. The people 
responsible for the contents did not have to wait until the software was ready. This meant a lot 
of gained time. Another advantage of this approach is, that maintenance is easier. Future 



changes, which are continuous, are anticipated, and can be implemented easily. If necessary by 
the client. 



The evaluation results 

The scores on the pre- and post-test are registered by the application. These results give some 
quantitative information about the increase in knowledge. It gives no qualitative information 
however about the application, or the perception of it by the participants. To get some more 
information about this aspect, all participants filled out an evaluation sheet. For each module, 
participants can evaluate four aspects: 

• The length of the module 

• The relevance of the content 

• The achievement of the objectives of the module 

• Overall quality of the module 

Each aspect could be scored on a five point scale [Excellent, Good, Fair, Marginal, Poor] 

Although there are some differences between the modules, the general rating is very 
satisfactory. The length of the modules is by 85% of the participants regarded as Good or 
Excellent. 

The relevance of the content of the modules is for some participants Tail" (15%), although still 
70% of them rated the content at Good or Excellent. Most participants (85%) thought that the 
modules achieved their objectives, and the overall quality was also for 85% of the participants 
good or excellent. 

The same questions are asked for the course as a whole, with more or less the same result. At 
this overall level, an additional question on the quality and appropriateness of the instructional 
materials and media is asked. The majority (55%) thought it was excellent. And 90% of the 
participants rated this question with Good or Excellent. 

These results are rather impressive. The objective is of course to achieve a 100% score, but the 
achieved percentage makes the day of the designers and developers. It is a high score in 
training. 

The participants had the opportunity tt> give their general remarks as well. Here, the general 
opinion is that there are some questions in the test that arc not clear. Most people think it is a 
good introduction for new employees, although some of the current employees think that it 
may be too much information in a short time-span. Participants tend to like the module on the 
history of Sun very much. The content itself and the fact that it is explained in a multimedial 
way is perceived as very interesting. 

Some people said that the course would not influence their motivation or attitude, because 
their interest is only with their own company. They did not want to be confronted with the fact 
that Sun International is a multinational company. On the other hand, some participants, e.g. 
from the bookkeeping department were very enthusiastic about these aspects. They were never 
confronted with these aspects, so it gave them a broader view on the company. 
Most participants (and this is supported by the resu!« on the tests) mentioned that they learned 
a lot. This is also true for participants who have worked with Sun International for some years. 
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Conclusion 

The evaluation shows that most of the needs were satisfied. The participants all know the core 
values as they state in their remarks. The structure of the organisation has become more clear. 
It gives participants a good perspective of the activities that the company performs. 
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Adaptable Educational Computer Simulations 



Willem Jaap Zwart, Albert FJ. Rhemrcv, Andr£ M. Berloth 



Introduction 

Using interactive computer simulations, learners can explore a domain by conducting 
experiments and observing the effects of these interventions. In this way the learner gains 
insight into the dynamic behavior of a system. Educational computer simulations consist 
of a mathematical model representing the real system and an educational representation of 
the system in which the possibilities to intervene and observe are realized (van Schaick 
Zillesen, 1990). The instructional strategy commonly used with this kind of educational 
software is 'discovery learning' or 'exploratory learning*. 

The use of this land of computer simulation is generally recognized as having great 
potential for learning and instruction (e.g. Reigeluth & Schwartz, 1989), because it is 
expected to motivate, to invite the learner to actively discover the subject matter and to 
allow for unlimited practice in applying known concepts and in discovering relations. 
Essentially, computer simulation is a dynamic visualization of a system in the real world. 
An important design issue in the development of interactive computer simulations is the 
selection of visualization techniques used in the program. The selected techniques should 
be adequate representations of the shown phenomena and must be understood and 
interpreted by the learner. Furthermore, the teacher, in his role as second author of 
educational software, should be able to change the visualization of the system in order to 
adapt the simulation to the needs of his students and to increase the correspondence with 
his other educational materials. 

In the Opical DataBase project (ODB-Project, Be^tebreurtje & Verhagen, 1992) an 
attempt is made to develop methods and techniques for the design of reusable multimedia 
databases. These databases should be suitable for multiple target groups and multiple 
instructional strategies. Therefore, the teacher is allowed to change the content of the 
database and the instructional patterns for the learner. The databases are multimedia: 
text, computer graphics, video, sound and interactive computer simulations are integrated 
into one system. 

The simulations implemented in this project allow for multimedia visualizations as well. 
An example of this kind of simulations is the renneting simulation (figure i). The 
purpose of renneting is the separation of cheese material in the milk (casein-micelles) 
from the liquid portion (whey). This can be accomplished by the addition of a specific 
enzyme which causes the casein-micelles to stick together. The process is influenced by 
several factors including pH of the milk, temperature and amount of rennet and amount of 
calcium chloride added to the milk. The model is capable of calculating the changes in 
the milk with time. The calculated entities involve the amount of casein left in the milk, 
the amount of para-casein, the amount of para-casein that is matted together and the 
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viscosity of the fluid. The last variable is the most important indicator of the progress of 
renneting. The aforementioned influences make up a complicated system, in which the 
behavior is not easily predicted. 




figure 1: the renneting simulation 



The following visualization techniques are used in this simulation: 

- graphs showing the change in variables 

- digits, showing the current value of variables 

- video segments used to illustrate the location of the process, the addition of the enzyme 
and to show tests which indicate the readiness of the renneting 

- audio segments played at certain threshold values in variables 

For the implementation of the computer simulations in the multimedia databases a flexible 
design system is developed using object oriented techniques. The working title of the 
system is OSS: Object-oriented Simulation System. An important part of the system is 
concerned with the selection of visualization techniques. The system allows for the 
connection of visualization tools to variables in the model. The selection of which tools 
to connect to a variable can be done by the teacher, prior to the use of the simulation, or 
can be done by the learner himself during run-time. This process of selection of 
visualization techniques can be done manually or can be executed with the help of a 
reasoning system. 

This paper describes OSS in general and the display-manager, the part of the system 
responsible for the selection of visualization techniques in particular. 




Instructional design for computer simulations 

Instructional design theories aim at prescriptions for the process of designing instruction. 
Although educational computer simulations are seen as an important instructional 
application, instructional design theories for educational computer simulations are scarce. 
Especially the choice of visualization techniques is a issue that seems to be underexposed. 
Winer and Vizquez-Abad (1981) try to develop a theoretical framework for educational 
simulations. They state that educational simulations should "represent the actual variety 
of the system under study" (p. 1 16). They observe that many simulations do not meet this 
requirement, thus severely constraining the range of interaction and the 'discovery' value 
of the simulation. They use the levels of knowledge representation of Bruner (symbolic, 
iconic and enactive), to describe the levels of interaction that learners should experience. 
These levels provide the learner "with a rich, integrated framework of knowledge on 
which to build a complete understanding" (p. 1 16). The prescriptions for the use of 
visualization techniques are limited to the statement that graphic or pictorial 
representations can be easily incorporated. 

The instructional design theory for educational simulations by Reigeluth and Schwartz 
(1989) focuses on a classification of educational simulations and the learning processes 
with these simulations. They distinguish: 

- procedural simulations 

- process simulation 

- causal simulations. 

These classes of simulations vary in learning goals and instructional strategies. The level 
of interaction allowed to the learners changes accordingly. In their view procedural 
simulations need the highest level of interaction and causal simulations the lowest. This 
level of interaction is expressed by the number of variables accessible to the learner. No 
prescriptions on the way of interaction are given. 

In the area of learning processes Reigeluth and Schwartz distinguish acquisition, 
application and assessment. In the phase of application generalization, automatization 
and utilization are important. All of these processes need adequate visualization, but 
except general statements on the relation between fidelity and the prior knowledge of the 
learner, no prescriptions on this are given. 

van Schaick Zillesen (1990) gives a general model of educational computer simulations 
(figure 2) 
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Model 
Communication 



Educational Representation 

figure 2: general mocel of educational computer simulations 

In this model the following parts are separated: 

1 a mathematical model 

2 an educational representation of the model 

3 an interactive communication layer between components 1 and 2 

The interactive communication layer arranges the following design decisions: 

the selection of variables to present to the student 

the way of presenting these variables 

the way the student can interact with the model 

the communication with other educational programs. 
Van Schaick Zillesen emphases the division of labor during the development of 
educational computer simulations. A team of specialists in the area of modeling, 
didactics, graphical design and educational technology should cooperate in this 
development process. 

He states (1991) that computer simulations have two main advantages above real 
experiments: 

visualization 

didactics. 

Visualization of abstract systems with a set of objects which can be observed and 
manipulated eases the construction of a mental model of the system. Although it is not 
clear from his model of educational simulation whether the actual visualization takes place 
in the educational representation or in the interactive communication layer his separation 
between model and representation is a useful construct in making computer simulation 
adaptable in this respect. 




In the second generation instructional design model (ID^ by Merrill, li and Jones (1991) 
a similar separation of domain knowledge and representation is proposed. The 
instructional transaction theory assumes that the lack of separation in current authoring 
tools is the main reason for the high development costs of educational software. They 
propose a library of 4 transaction shells* which are reusable software components suitable 
for a certain type of educational goal. Every transaction shell consists of an implemented 
abstract instructional algorithm. This abstract algorithm can turn in to a concrete 
educational application by filling the transaction shell with: 

a knowledge base, a system independent description of the subject matter 

a resource database, containing system dependent representations of the subject 

(e.g. pictures, sounds etc.) 

and 

a database with instructional parameters, describing for instance the sequence of 
the subject matter components and the instructional strategy used for these 
components. 

Each database should be created by a specialist The knowledge base by a subject matter 
expert, the resource database by media experts and the database with instructional 
parameters by an educational technologist For the construction of the knowledge base a 
syntax is given (Jones, Li & Merrill, 1990) consisting of three types of frames (entities, 
processes and activities) with four kind of elaborations (attributes, components, 
hierarchies and associations). Examples of instructional parameters are given for one 
instructional algorithm. Every transaction shells should however have its own 
instructional parameters. Furthermore, it should be possible to change the parameters 
during the delivery of instruction based on student model. Sample instructional 
parameters in the 'portray knowledge* group are concerned with the representation of the 
system (Merrill, Li and Jones, 1992): 

view (with possible values structural, physical and functional) indicates the 

principal way the knowledge is represented 

mode (with possible values language, symbolic and literal) refers to the nature of a 
view 

fidelity (with as possible values the range [low high]) refers to how close the 

representation resembles the real thing. 
No prescriptions for the content of the resource database are given. 

Applied to educational computer simulation and integrated with the model of van Schaick 
Zillesen the following match can be made (Zwart, 1992). For educational computer 
simulations: 

the knowledge base should contain a mathematical model of the system 

the resource database should include certain representation techniques to visualize 

the behavior of the model 

the instructional database should included parameters for the selection of 

appropriate visualization techniques. 
From this it is clear in which part of a flexible educational computer simulation system 
the selection of visualization techniques should take place. 

Collis and Stanchev (1992) pay attention to visualization, interactivity and intelligence as 
trends in computer based educational simulations. Although their paper addresses these 
issues in the context of micro-computer based laboratories (MBLs) the trends are 
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applicable to educational computer simulations based on mathematical models as well. 
With respect to the visualization in computer simulations they observe an increase in 
quality and quantity of visualizations. The speed of appearance and the graphics in 
simulation software are improving enormously. Integration of simulation with interactive 
video is used more and more. Tney distinguish five major types of visualization: 

still and moving graphics 
sketches and drawings 
digitized photographs 
animations 
moving video. 

Collis and Stanchev sketch a cognitive-instructional grid including simple to more 
complex cognitive skills and different instructional strategies ranging from motivating 
learners to stimulation higher order thinking. In the mapping of visualization techniques 
on this grid they assume that 14 more and moving is better in terms of visualization, ... no 
matter where one is on the cognitive-understanding grid" and that the more complex the 
instructional task the more complex visualizations are desirable. These general 
assumptions are then refined upon by the hypothesis that complex visualizations are 
particular useful for orientation and motivation (simple tasks) and for higher order 
thinking (complex tasks), but the more simple visualizations are suitable for enlarging 
knowledge and problem solving ("middle" tasks). 

Min (1992) states that K a complete simulation learning environment consists of a series of 
elements, the most important of which are: 
highly visualized, graphical output 

highly visualized, conceptual, underlying mathematical models...." (p. 177). 
He argues that different kinds of visualization and presentation should be presented to the 
learner parallel. As visualization techniques he names a conceptual scheme of the model, 
output curves and "video-messages" as feedback. The wide range of different kinds of 
visualizations range from abstract to concrete. However, no prescription on the selection 
of these techniques is given. 

We can conclude that current instructional design theories give little explicit prescriptions 
on the selection of visualization techniques in educational computer simulations. We 
doubt if in the future general applicable prescriptions will appear. Too many 
characteristics of domain and learner are involved in this selection. The rather limited 
results of research in the area of interaction between treatment and aptitude directs us 
towards more flexible learning environments in which design decisions can be made in a 
later phase of the life cycle of educational software. The best thing to hope for in the 
near future are hypothesized guidelines, which need to be proofed by experimental 
research. 

Consequently, our system for the development of educational computer simulations is 
flexible in the choice of visualization techniques. 
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The system allows for adaptation, both in the preparation of a simulation by a teacher and 
during the use of the simulation by the student, in: 

which variable to display 

which parameters can be changed 

how parameters and variables are visualized 

what should happen during the simulation run 

which model to use 



Set-up of the system 

The goal of our Object-oriented Simulation System (OSS) is to ease the development of 
adaptable interactive educational computer simulations, that can be integrated within 
multi-media databases. Therefore, OSS has a strong separation of tasks over software 
components, which can be configured separately. The communication between the 
components is standardized. Components can be implemented in different programs, 
using multi-tasking techniques. Although it is not the main development goal of OSS, the 
combination of a strong separation of tasks and a standardized communication protocol 
allows for flexible configuration of components into a simulation environment as well. 

Our system is divided into three main components: 

the model manager, which handles calculation 

the interaction manager, which handles the interaction and visualization 

the communication manager, responsible for all communication between 

components. 

The whole system is managed by the simulation manager. Furthermore, other programs 
can communicate with the simulation. For example an instruction manager can change the 
simulation during run-time or deliver additional domain knowledge. This allows an 
instructor to interact with the model and the students. 

The minimal simulation environment consists of one model manager, one communication 
manager and one interaction-manager. However, it is also possible to let more models 
communicate with one interaction-manager, to let more interaction managers display the 
results of the models or to add other managers to the environment All the different 
components talk to each other via the communication-manager: they only need to 
understand the language the managers use. The separate components of a s i mul a ti on 
environment need not to be running on one and the same computer. The communication 
implementation allows components to communicate over a network. This allows for 
instance more than one student to work on the same simulation. 
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Simulation Manager 





Interaction Manager 



Learner Model 



Display Manager 



Instructor 



figure 3: general set-up of OSS 

The interaction manager is the only manager that communicates with the user. For this 
purpose the interaction manager has 'interactors'. Each interactor has one input or output 
function. Every interactor has attributes that can be changed like its location, its color, 
its name etc. 

The system is designed using Object Oriented Techniques and developed on a Apple 
Macintosh using C++ and MacApp. The communication system is implemented using 
Apple Events. 

An example of non-standard component that can be used is the scenario-manager. This 
manager adds dynamic to the simulation. This manager makes it possible to let the 
simulation environment take actions triggered by conditions. Sample conditions included: 

the value of variable 

the change of a variable 

the comparison of two variables 
Actions that can be executed included: 

the change of the value of a variable 

the execution of a model command, like Start, Stop and Pause 

the change of an attribute of an interactor 
The information the scenariomanager needs is available in the communication protocol. 
The developer of the simulation simple makes a file with the actions and conditions and 
the simulation will behave accordingly. 
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figure 4: sample simulation scenario 



The interaction manager is capable of displaying a variable using some 30 kind of 
interactors, including VLP-stills and running video, audio-feedback, the normal graphs 
and indicators, animated input etc. 



Adapting the representation 



The display manager (an extension of the interaction manager) is the component of the 
system responsible for the selection of visualization techniques. 
Its first main task is to add, delete and change interactors on the screen and to connect 
them to variables in the model. This task is implemented by giving commands, using the 
communication manager, to other parts of the system. 

This mechanism allows the user or teacher to select a specific visualization technique for 
a specific variable directly. 

Besides this direct interface a reasoning mechanism is implemented that can be used to 
select the most appropriate visualization technique based on characteristics of the domain, 
the learner and the visualization technique. By using characteristics a more abstract level 
of selecting visualization techniques is realized. The developer does not need to specify a 
visualization technique for every variable/learner combination, he just specifies the 
characteristics. The system is than capable of advising the user in the choice of 
visualization techniques. Figure 5 outlines the way this is implemented. 
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figure 5: outSne of selection aigorthm 
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The interactors, learners and variables are all characterized on the following items 
(Zwart, 1992): 



Output-mode (enactive, iconic, symbolic) 
Order of presentation (zero, first, quantitative) 
Time-dependency (momentous, dynamic) 

The output-mode variable is based on the representation levels of knowledge by Bruner. 
For interactors it indicates the level of representation used as visualization technique. For 
learners it indicates the level of representation the learner can understand. For variables 
it indicates the preferred way of representing the variable. 

The order of presentation variable is based on research by White & Frederiksen (1989). 
It describes the form in which relations are shown to the learner. In the zero order 
presentation only the presence or absence of a phenomenon is shown. In the first order 
presentation the direction of change in the phenomenon is shown and in the quantitative 
level the exact value of the variable is shown. 

Learners, variables and interactors can have more than one value of a characteristic. For 
instance learners can be able to interpret both enactive and iconic representations or a 
variable can be shown momentous and dynamic. 

Visualization techniques come in two forms. The first are standard techniques, that do 
not need additional (domain dependent) resources. Examples of these standard techniques 
are graphs, bars etc. All that is needed to use these techniques for displaying a certain 
variable is a connection between the interactor object and the variable. The 
characteristics of these techniques are set within OSS. The other land of techniques do 
need additional resources. Examples are video segments and animations. Every 
visualization of this kind needs to be labeled separately. 

The list of characteristics can be expanded. 

Currently the algorithm is binary: a visualization techniques is either suitable or not 
suitable for this learner/domain combination. However, using the same labeling of 
visualization techniques, learners and variables with another algorithm a ranking of the 
appropriate techniques can be realized. 

The current implementation of the display manager allows the user to change or add a 
visualization technique for a variable. Whenever a selection of visualization tools is 
needed the reasoning system builds a list of appropriate techniques, from which the user 
can select one item. Figure 6 shows a sample outcome of the selection process. 
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figure 6: Selection window of the cfisplay manager 
Conclusions 

The techniques implemented in the display manager of OSS provide a semi-automatic tool 
for the selection of visualization techniques. The labeling of learners, visualization 
techniques and elements of the domain allow the developer of educational computer 
simulation to specify attributes on a higher level. The characteristics can be seen as 
instructional parameters as inttoduced in the Instructional Transaction Theory. 
With the absence of a prescriptive theory of visualization this system realizes the 
flexibility needed for adaptable educational computer simulations. 
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A look at Multimedia 
Jan Evers 



Abstract 

Multimedia land Ls not getting any clearer, lately. More and more the innocent PC-user runs 
into names like Multimedia PC, Video for Windows, Indeo, CD-I, Quicktime and DVT. What 
do these names mean, and what possibilities do they offer? This paper attempts to give an 
insight in the world of multimedia by describing the most significant possibilities of the use of 
multimedia. 

The application of multimedia requires a high capacity delivery system. Therefore, firstly the 
definition of multimedia applications and the display system requirements will be discussed. 
After that, a description of the most important multimedia platforms will follow: Multimedia 
PC (MPC) with Video for Windows and DVI, Apple Macintosh with Quicktime, Philips' CD- 
I. It is expected that MPC will play the largest role on the market. Therefore the final part of 
this paper will investigate the possibilities of this platform. 



What is multimedia ? 

A multimedia application is defined as a computer application in which more than one of the 
following information or media types are integrated: text, graphics (animations), sound and 
video. A lot of the existing computer applications present both text and graphics. They are not 
called multimedia, however. Multimedia usually is not recognized as such until audio and/or 
video are being used. 

The high demand on computer hardware is a result of the application of audio and even more: 
of video. The amounts of data are that large (video: approximately 10 Mb per minute) that the 
need for faster computers with more internal memory (RAM) and more storage capacity 
increases. A PC based on a 80386 processor, 4 Mb RAM and 200 MB hard disc is not 
extravagant. Moreover, the presentation of acceptable quality audio and video requires 
additional hardware, existing of boards. 

The requirements for hardware used for development of multimedia requires even higher: with 
these computers audio and video delivered in an analog way from CD or videotape have to be 
converted into a digital representation on the hard disc. Acceptable development hardware is 
equipped with e.g. a 80486 processor, 8 Mb RAM and 1 Gigabyte hard disc. Quite often the 
digitising of sound and video requires also additional hardware (boards). 



Different platforms 

A large number of platforms for displaying multimedia applications is available on the market. 
The most important multimedia platforms vill be discussed here, after that the PC-platform 
will be described in more detail. At the moment the most important platforms are: Multimedia 
PC (MPC) with Video for Windows and DVI, Apple Macintosh with QuickTime, CD-I by 
Philips. 
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The main difference between CD-I and the other platforms mentioned is the fact that CD-I 
uses a very specific display device: the CD-I system. This system can also be used to play-back 
audio CD's and Photo CD's (Kodak). Connecting the CD-I system to a television set is 
necessary to view a CD-I application. The CD-I application is operated by using a joy stick or 
infrared remote control. A keyboard is not incorporated. A CD-I application is stored on a 
CD, which implicates that it is not possible to update or replace e.g. a part of the video in the 
CD-I application. In order to do this, the entire CD has to be produced and delivered once 
again. 

A multimedia application developed for MPC or QuickTime can be displayed using a standard 
platform (PC respectively Macintosh) under a standard operating system (DOS/Windows 
respectively System 7), which means that ail the other facilities offered by those systems are 
available. Examples are wordprocessing facilities, database management, etcetera. Because of 
the fact that the MPC and QuickTime both use the standard file structures of the platform, it 
does not matter if these files are stored on hard disk, CD-ROM or on a network. Putting all 
files for a multimedia application on hard disk offers the advantage that replacing those files by 
new versions is easy. In this way the updating of e.g. audio or videoparts is very convenient. 
An other advantage of using the hard disk in stead of a CD-ROM is speed. The displaying of 
sections of video can start much quicker when using a hard disc than when these fragments are 
stored on CD-ROM. Also, because of the higher speed of the harddisc, video played from 
hard disk often can have higher quality, at the cost of more discspace. Using CD-ROM limits 
the user to a maximum of about 10 Mb of video per minute. Extracting larger amounts of data 
per minute is not possible, because these amounts can not be read fast enough from CD-ROM. 
Recently double end quadruple speed CD-ROM players came available, which make play-back 
at higher data rates possible. 

The advantage of CD-I is the relatively low priced display device: about f 1500.- A good 
quality multimedia PC will cost approximately f 6000.- to f 10,000.- (when equipped with a 
large hard disc, 80486 processor, 4 Mb RAM, perhaps a DVI-board). This PC can also be 
used for other purposes, however. 

The investments that have to be done to develop multimedia applications using CD-I 
technology are much higher than the investments needed to work with both other platforms. 
Software tools for CD-I cost thousands (or tens of thousands) of guilders, while the costs for 
tools for the other platforms are significantly lower. Designers Work Bench (DWB) by Scrypt 
Systems Inc. is an example of a complete development system for CD-I. The system including 
all required hardware and software (authoringsystem) costs f 86,850.- The authoringsystem 
offers possibilities to add C-routines. 

The CD-I development process is complex. Digitized files have to be made using a PC, Mac 
or Sun. After that the files have to be converted into CD-I fileformats. Then the CD-I 
application can be developed using a CD-I programming environment. Finally, all digitized 
files and the application have to be merged and tested with CD-I emulation software, disc-set 
up software and test software. This final step often requires expensive extra hardware. When 
the application has been developed, a master tape has to be made to produce the actual CD- 
I's. 

Full screen, full motion (screen filling and 25 frames a second for PAL, the European tv- 
system; 30 frames a second for NTSC, the American tv-system) video to be used with CD-I 
nowadays can not be produced in house yet. The videotape has to be sent to a third party, for 
digitizing. 
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Multimedia on PC 



Although Macintosh with QuickTime is this moment's most advanced platform, the 
Multimedia PC (MPC) market share is rising fast. Without any doubt, the MPC market will be 
a large, if not the largest, market for multimedia applications. For a major part this is caused 
by the tremendous effort Microsoft is putting in maJcing a success of the MPC. At this time 
Macintosh with QuickTime mainly differentiates in offering better tools to the multimedia 
application developers. Making animations and adding video still is much easier on the 
Macintosh compared to the MPC. However, this seems to be a matter of time. 

The minimum technical specifications for the MPC have been defined by Microsoft, IBM and 
Intel, among others. The specifications are: 10 MHz 80286, 2 Mb RAM, 30 Mb hard disc, 
CD-ROM device, VGA video board, (16 olours), audio board, mouse, Windows with 
multimedia extension (standardized in Windows 3.1). Note that these specifications really are 
the minimum. Efficient usage of Windows requires a 80386 processor, combined with 4 Mb 
RAM. The usage of video requires even more: 80486, 4 Mb RAM, 200 Mb hard disc and 
VGA board with 256 colours. The Windows multimedia extension offers very easy 
possibilities to create and display digitized audio. 

A CD-ROM device always is incorporated in the sale of a Multimedia PC or a so called MPC 
upgrade kit. When you decide to upgrade your PC to be able to work with multimedia, it is 
strongly recommended to buy & CD-ROM device. More and more the multimedia applications 
are delivered on CD-ROM, because of the large amount of data they consist of. Often the 
possibility exists to install an application from CD-ROM instead of from floppy discs. Also 
data intensive applications, such as manuals and (multimedia) dictionaries, are often delivered 
on CD 10M. 

Because 10 floppy discs for one application no longer is exceptional, installing from CD-ROM 
is much more convenient. Installing those amounts of data on hard disc is not recommended. 
All in all buying a CD-ROM device is worth the while, especially taking into account the 
decreasing costs. By now prices have gone down below fl 1000.- 

In the beginning of 1993 Microsoft introduced Microsoft Video for Windows, an extension of 
the multimedia possibilities of Windows 3.1. Video for Windows allows the display of 
digitized video without the need for extra hardware in the PC. This is called Software-only 1 
video. (The play-back of the sound accompanying the video requires an audioboard, 
however.) The quality of the video that is being displayed, depends on the powerfiilness of the 
display device: a smaller video window and less frames per second on a less powerful display 
device. This means that on a 80486 PC 25 frames per second can be displayed in a 160x120 
pixels window. In a 320x240 pixel window (4 times as big) only 10 frames per second can be 
displayed. For 'software-only 1 video a VGA board with 256 colours is the minimum. The 16 
colours of a normal VGA (board) offers no acceptable video quality. 
A developer of Video for Windows applications has to buy Video for Windows (fl 500.-). The 
tools for digitizing video are included in the package. The libraries required for displaying 
Video for Windows files are distributed freely with the application developed. Therefore it is 
not necessary for the user to buy extra software to display Video for Windows applications. 

Video for Windows offers the possibility to install other compression/decompression 
algorithms (codecs). An example is the Indeo codec by Intel. In displaying files made with 
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Indeo Video for Windows rses Intel's DVI (Digital Video Interactive) delivery board, if 
available. If this is not the case, the Indeo files are displayed by the processor (e^ 80486). 
The size of the display window and the number of frames per second are being adjusted to the 
power of the processor available. If the DVI delivery board can be used, the Indeo files can be 
displayed full screen , full motion (640x480 pixels, 25 frames per second). The sound is 
always displayed by the special MPC audio board, regardless of the availability of the DVI 
board. A disadvantage of Indeo is the fact that it is not possible to display stills (yet). Stills are 
digitized pictures, slides or pictures from a video. They need a higher resolution than a 
videoframe, because the eye is more critical for stills than for moving video. 
A DVI delivery board, the Action Media II Delivery Board, costs approximately fl 2500.-. 
Ace Coin in Great Britain markets a compatible board, the PCTV-SLC, for approximately fl 
1400.- Digitizing video for both boards requires the Action Media II Capture Board. This 
board will cost around f 1000.-. 

The maximum resolution Indeo video under Video for Windows can offer is 320x240 pixels. 
When higher video quality is demanded, it is possible to use the DVI delivery boards with the 
DV-MCI driver under Windows. By doing this, no use is made of Video for Windows and the 
accompanying file formats. The video, audio and stills (possible when using DV-MCI drivers) 
are being saved in DVI-files (so called AVSS-files). The highest quality video that can be 
presented in this way has a 512x480 pixel resolution, and is called Presentation Level Video 
(PLV). An advantage of using DVI with DV-MCI is that an extra audioboard no longer is 
required. Sound is being played from the DVI delivery board. A disadvantage is the 
impossibility to make PLV video files by yourself. Analog video, for example on a Betacom 
SP videotape, has to be sent to the video production company Teletota in Paris. Teletota is 
Intel-licenced to make PLV. To do this, it uses a large computer, that needs about 100 
minutes of computing time to digitize and compress 1 minute of video 

Digitizing and compressing 1 minute of video costs $ 250.-. However, it is possible to 
compress and digitize the video in house using the Action Media II Capture Module. This 
results in Real Time Video (RTV). RTVs maximum resolution is 256x240 pixels. The Indeo 
video quality (340x240 pixels resolution) is slightly better than RTV video quality. 

Under OS/2, Presentation Manager, IBM offers a platform that is comparable to the 
Multimedia PC with Video for Windows platform: Ultimedia. This is a PS/2 system in which a 
DVI delivery board and a CD-ROM XA (extended Architecture; an extended version of CD- 
ROM) are incorporated. Here the multimedia extensions on Presentation Manager are 
available to the developer. 
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Concluding remarks 

The Multimedia PC with Video for Windows plays an important role on the current 
multimedia market. Microsoft puts every effort in supporting this platform and making it a 
success. Especially now that it is possible to use DVI under Windows, this platform is a good 
alternative for applying multimedia, both from a user's and a developer's point of view. 
Applying multimedia requires powerful hardware, however. The defined minimum 
specifications actually are not sufficiently powerful. Recommended is a 33 MHz 80486 PC 
with 8 Mb internal memory (RAM), 200 Mb hard disc and a CD-ROM device. For a 
developer's system a 1Gb (Gigabyte) hard disc is recommended. 
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QuickTime 



Jakob Sikken 



Introduction 

This paper will discuss the possibilities of QuickTime in using digital multimedia. Digital 
multimedia gives you possibilities of using multimedia, that are not available with 
traditional multimedia (video and sound from an analogue source, such as laser disk player 
and the video recorder). It gives you instant access to every pan of your multimedia 
material arid you can combine pans to create the material you need for that student at that 
specific moment. 

Data amounts 

To make working with digital multimedia at a reasonable cost possible, image 
compression is needed. One screen image (640*480 pixel's) in 24 bit colour contains about 
900 KB of data. Displaying one second moving video at 25 frames per second (PAL), this 
gives about 23 MB of ..*a per second. One minute of moving data will need about 1318 
MB of data. Only ver^ expensive computer equipment can handle this amount of data per 
second. Luckily moving video can be compressed very well without losing to much image 
detail. 



QuickTime 

QuickTime is a architecture 

QuickTime is a platform-independent architecture for multimedia. Because QuickTime is 
an architecture and not a program, it is not the question if should you use either 
QuickTime or an other program, let's say AuthorWare. But you use both, 
AuthorWare A C(or something else) for the instruction part and let QuickTime handle the 
multimedia stuff. QuickTime adds the capability of working with multimedia to the system 
software. This makes it easy for every program to use multimedia, not only for 
commercial programs, but also for programs that you develop. 

The QuickTime architecture performs four key functions: 

1 synchronising a time-based data stream; 

2 compressing data to minimise storage needs on the user's computers; 

3 providing a standard graphical interface for creating, editing, and navigating through 
movies; 

4 establishing a standard file format that accommodates dynamic information. 
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QuickTime can consistently play its multimedia files (called movies) at the highest 
possible quality, keeping sounds, images and colours properly synchronised on everything 
from low-cost personal computers to high-performance work stations. 

QuickTime's modular design allows new capabilities to be added as quickly as they are 
developed. For example, as new methods of image compression are developed, you will 
be able to use them without any changes to the applications, simply by adding the new 
image-compression components. 

Movie has a number of channels 

A QuickTime movie contains a number of channels. There are three different types of 
channels: video channels, sound channels and command channels. A movie can contain 
any number of the three different types of channels. A movie does not need to contain 
moving video, it could just contain sound or only a command channel. 
This makes it possible to have a sound channel for each language you want to have. You 
can use separate video channels for each type of screen, such as a black and white 
version, a Gray version and a colour version. An other possibility is different video 
material for different experimental conditions. 

In a command channel you can put commands to synchronise actions of the computer with 
the video and/or sound materia'. 

What can it do 

Play, create and edit movies 

Using QuickTime you can play movies, but also create and edit movies. QuickTime itself 
contains the basic stuff to create and edit movies, such as compressing and an interface for 
digitisers. The QuickTime starter kit for the Macintosh contains a program called "simple 
player", in which you can play and do some simple cut and paste editing on movies. But 
if you want more control and editing possibilities, you need a specialised program for 
creating and editing movies, such as Adobe Premiere or Diva VideoShop. 

Still images 

Multimedia is not only about moving video, but it can also be plain non moving video 
(still images). Where moving video consist of very large amounts of data, a number of 
still images can also be a large amount of digital data. 

A part of QuickTime are the CoDecs or the components for compression and 
decompression of images. You can use these CoDecs both for compressing moving video 
and for still images. This gives you the possibility to store and therefore use a large 
amount of still images. 

Photo-CD 

An excellent example of medium containing a large amount of still images is the Photo- 
CD by Kodak. QuickTime makes it possible to access all the images on the Photo-CD in 
all the available resolutions. For the Macintosh the Photo-CD contains a lot of picture files 
neatly organised in different folders for the different sizes. The icon for each picture file is 
a small image of every picture. 
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Hardware needed 



Currently QuickTime is available on Macintosh and Window PC's. It is announced for 
Silicon Graphics work stations. 

Macintosh 

QuickTime for Macintosh works with all 68020 or later Macintosh computers that have at 
least 2 MB of memory and are running either System 6.0.7 or any version of System 7 
(with at least 4 MB of memory). This means that most Macintosh models, but not the 128 
and 512 KB Mac, Fat Mac (Lisa), Mac ED, Mac Plus, Mac SE, Mac Portable, the Mac 
Classic and the PowerBook 100, can use QuickTime. All these models are out of 
production for at least a half year. 

The faster the hardware is, the better the performance of QuickTime will be. But a 
essential part of playing a movie, the synchronisation of sound and video, will be kept on 
every Mac. It will skip frames to keep the sound and video synchrone. 

Most of the available hardware to accelerate display of moving video will work under 
QuickTime. In the worst case you will have to recompress your movie to the compressed 
data format of the hardware accelerator. 

The current version of QuickTime for the Macintosh is 1.6.1. This version has some bugs 
fixed of the 1.5 version, special decompression code for PowerBook displays (16 Gray 
shades) and a reduces memory usage. QuickTime for Macintosh is distributed free of any 
costs. 

IBM PC's and compatible's 

In our laboratory we have no experience with QuickTime for Windows yet, so the 
following information comes from the documentation (see reference list). 

QuickTime for Windows runs on any 80386-based or higher system that has at least 4 
MB of memory, an 80 MB hard disk, and a VGA or better graphics adaptor. Movies that 
include sound require a sound card. System software requirements include MS-DOS 5.0 
and Windows 3.1 or later. 

QuickTime for W indows consists of a set of Dynamic Link Libraries (DLLs) that provide 
all the functions necessary to play QuickTime movie files in Windows-based applications. 
QuickTime for Windows also provides Windows applications with the ability to display 
still images contained in Macintosh-created PICT files. 

The latest version of QuickTime for windows is 1.1. This version contains the CoDec for 
using the compact video compressed data format. On the Macintosh is that the best one 
for normal video. It also supports OLE 1.0. QuickTime for Windows is distributed along 
with a number of QuickTime-aware applications for Windows, 
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By using QuickTime-compatible applications on both Macintosh and Windows system, 
users can interchange movies easily and enjoy a consistent user interface for working 
with multimedia. With hundreds of applications already supporting QuickTime for 
Macintosh, user can create and edit movies on Macintosh systems, then share them with 
colleagues, clients, and friends who have IBM-compatible systems. 

The same movie can be played on both Macintosh and Windows systems, provided that it 
is self contained and has been set to be playable on non-Apple computers. Movies can be 
set for multiple platforms when they are created, or they can be converted using the 
Movie Converter software in the Macintosh QuickTime Starter Kit. 

Software 

To use the QuickTime functionality you can use QuickTime-aware applications. These are 
applications with build-in code for using QuickTime. Examples of such applications are 
Word Perfect, Microsoft Word, PowerPoint, AuthorWare and MacroMind Director. 

For some other applications are external commands available to use QuickTime, such as 
HyperCard and SuperCard. 

And for Macintosh applications that can handle pictures, there is an interesting system 
extension (or init) called "Wild Magic*\ This extension makes it possible to copy and 
paste QuickTime movies, while the application thinks it's handling a picture. 

All applications, which fit the above mentioned descriptions, can play a movie with the 
standard graphic user interface. Some applications can do a lot more, such as controlling 
the playing of the movie from the application. It is possible to play only a part of the 
movie at any desired speed (within the hardware limitations) and direction. An other 
interesting option is to play a number of parts of different movies directly after each other, 
thus giving you the possibility to compose your movie at run time. 

Creating movies 

There are four basic steps in creating a QuickTime movie. 

1 Digitise; 

2 Edit; 

3 Compression; 

4 Adjust to storage medium. 
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Digitise 

With digitise I mean digitising a video signal that comes from a video-camera or video- 
recorder. In this case you can use existing video material on tape. QuickTime offers a 
software interface to the digitiser and the program that controls the digitising process. 

It is also possible to create a movie without a digitiser and let it be generated by the 
computer. This could be a good way the create an animation or a "movie screen dump". 

Edit 

After you have digitised your video material or generated it, most times it must be edited 
to get the right movies. Using editors you can cut t copy and paste the right video 
fragments in the right sequence. During this process you can use a variety of different 
transitional effects and special effects on the video and sound material. Adobe Premiere is 
such a QuickTime editing program. 

Compression 

As I have explained in the beginning uncompressed digital video gives you a very large 
amount of data to handle. This amount can be acceptable during the edit process, but in 
most cases it will not be acceptable on the target machines. So the digital data has to be 
compressed. 

Adjust to storage medium 

Just compressing the digital data might not be enough. This depends on the storage 
medium you intent to use. There are two aspects of this medium important for the 
compression methods. First is the data transfer rate of the medium. For most CD-players 
this is 150 KB per second, you can't get the best results if your movie needs 500 KB of 
data per second. The second aspect is the total amount of data you can store on your 
storage medium and the total number of minutes of movie you whish to use. Both aspects 
results in a desired number of KB data per second. 

To reach the desired transfer rate, some times the movie has to be more compressed. 
There are a number of ways to get a higher compression. First there is the possibility to 
just compress it more by accepting a lower quality image. You can leave images al to 
gather out of the movies, resulting in a lower frame rate. An other way is to reduce the 
image size. Each of these ways have a lower quality result in common. 

Expensive way of working 

In the expensive and the fast way you do the digitising and the compression real time. To 
do this you need additional hardware for the compression. This hardware is expensive, but 
you save some money on the needed disk space and the time needed the create the movie. 
Indeo Technology of Intel can do this task, but only if you accept a lower image quality. 

For the best possible image quality, with Indeo Technology t you need to send the video 
tape to a company for digitising and compression. This will cost you a few hundred 
dollars per minute of video. The Indeo Technology is available for both Macintosh and 
IBM-PC computers. 
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Cheaper way of working 

A cheaper way of working is to do the digitising and the compression after each other. 
This can be done in two ways, the first way is to grab a single frame and then compress 
it. After it is compressed, grab the next frame and so on. This is a good way of working if 
you video material can be computer controlled accessed on a frame by frame base. This" is 
true for the laser disk and for expensive video recorders (and the video tape must be 
prepared this operation). 

The second way is to digitise the running video and transfer to data real time to the hard 
disk. Due to the limited transfer rate of hard disks and the channel to it, the data stream 
must be reduced. There a few ways of achieving this, do not use the full screen and do 
not use every frame. Working this way you can achieve reasonable results, for example an 
image size of 320*240, 16 bit colour (32000 colours) and 10 frames per second which 
gives a data stream of 1.5 MB per second. 

An image size of 320*240 is only a quarter of the standard VGA or Mac II screen. 
But in the reality this is the highest achieved image size, even the DVI boards on the 
IBM-PCs and the EYQ boards for the Mac (both Indeo Technologies) use this image size. 
To get it full screen the image is enlarged. This is due to the way most digitisers work, 
they only capture one of the two fields out of a frame. A PAL video signal has 25 frames 
per second and each frame exists out of two fields. Each field skips every other line, it has 
only half the number (312 and 313) of lines a frame (625) has. This gives an effective 
refresh frequency of 50 images per second instead of 25 images. 



Experiences 

Playback experiences 

We have some experience with QuickTime for Macintosh. We have found out that 16 bit 
colour (32000 colours) is necessary. The colour resolution of most computer displays, 256 
colours to be chosen out of a pallet of 16 million colours (24 bit), is not enough for a 
good result. There are just not enough colours to make a good image. Therefore 16 bit 
colour (32000 colours) is not a luxury but essential. 

From our experiences we can say that a movie with the size of 320*240 pixel's (a quarter 
of the screen) can be played with frame rate of 10-15 frames per second on a Mac Ilfx. 
This is done with software only and played from a hard disk. This means that it can also 
be done using any mid and high range Macintosh (Centris and Quadra line). We have no 
experiences with a LC III, but we think it will not achieve this performance. And of 
course if you choose a smaller image size you can use a slower Mac or get more frames 
per second. 

To achieve good results it is necessary to use high quality video material. High quality 
video material has less noise in the image and therefore less unneedcd work in playing 
back this noise. If there is much difference between two frames, playback frame rate will 
drop. So a movie where a large part of the image is changing will give a slower playback 
frame rate. 
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Most of the Macs (Mac lis and Mac LCs) in our faculty are to slow to display reasonable 
size movies and can't display enough colours. An image size of perhaps 200*150 pixel's 
will play fast enough, but for most purposes the 256 colours the computers can display are 
not enough. We suspect the are the same applies for the IBM-PCs with QuickTime for 
Windows or Video for Windows. 

Storage mediums 

The playback performance of QuickTime depends also on the storage medium used. The 
faster the storage medium is, the better the playback performance is. It is better to use a 
storage medium that has a faster data transfer rate then actually needed. The higher 
transfer rate gives the Mac more time to decompress the movie. This means that a hard 
disk is better storage medium then an optical disc or a CD-ROM. Optical discs and CD- 
ROMs can very good be used as storage medium but you will get a lower quality movie 
then could be expected based on the data transfer rate of the storage medium. 

Recording experiences 

During our work with QuickTime, we have experienced that you need a lot of disk space 
during the creating and editing of the movies. Compressing a movie is a lot of work for 
the computer, we found times of one hour or more computing time for one minute video. 
You need a very fast computer for making movies. QuickTime movies are, despite the 
compression, large files something like 10 MB per minute video. 

Recording a movie in slow motion is a possibility, but you need a video recorder capable 
of playing perfect slow motion. That type of video recorder is very expensive, you are 
talking about professional equipment. 



Examples 

By using QuickTime you get a lot more possibilities than with traditional video equipment 
(laser disk player or video recorder). In a limited way you can compose your movie 
runtime, from different parts from one or more movies. 



An interesting example that was showed, was a kind of simulation where you stood on top 
of the Golden Gate bridge (over the Bay of San Francisco). You could move the camera 
in every direction you wanted. The base of this example was a QuickTime movie 
containing camera shots for every possible direction. Depending on the next camera 
direction, the corresponding frame number was calculated and shown. Thus giving you the 
illusion of looking in every direction you wanted. 
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Conclusions 

QuickTime for Macintosh as a software only solution can be good used with certain 
restrictions. 



. A quarter screen size; 
. 10-15 frames per second; 

. Fast enough Mac capable of 16 bit colour (32000 colours) display (Centris and Quadra 
line); 

. High quality video material; 
• Steady image. 

And if you can not live with these restrictions, you can buy additional hardware to achieve 
a better result. More frames per second, Tall screen size video, and this all even on a 
slower Macintosh like the Mac II. 

In the nearby future there will be Macintoshes with additional hardware for handling 
multimedia standard build in. These computers will not only have video and sound 
outputs, as today's computers have, but also video and sound inputs. 

References 

Adobe Systems Inc. (1992), Adobe Premiere™: User Guide, Adobe Systems Inc., 

Mountain View, USA. 
Apple Computer, Inc. (1992), QuickTime for Windows Programmer's Guide, Apple 

Computer, Inc., Cupertino, USA. 
Apple Computer, Inc. (1993), 15 Steps to Better Movies, Apple Computer, Inc., 

Cupertino, USA. 

Apple Computer, Inc. (1993,2), Inside Macintosh: QuickTime, Adison-Wesley Publishing 
Company, Amsterdam, the Netherlands. 



150 



143 



Conclusions 



Ivan Stanchev, Jan Schoenmaker 

This report is deliberately built up around instructional visualisation and not upon multi media. 
Of course there is a close relation between the two, but in general when talking about multi 
media , we mean only the technology of storing , retrieving and presenting text, data, graphics, 
animations, audio and video. Our focus is more on concepts of visualisation , where to use it 
and how to design it. 

We will possibly extend the scopo in the near future towards virtual reality. Virtual reality 
allows one not only to perceive a pseudo real world as with multi media, but also to act upon 
it to a far more realistic level than with the possibilities of interaction which we have available 
with our regular computer technology. They are restricted to operating different input device^ 
like for instance mouse and touch screen which are of course 'artificial' devices of action 
compared to direct manipulation by hands, feet or head. 

Generally speaking our conclusions are twofold regarding future research and 
development: 

Focus on conceptual aspects and types of applications, rather than on multi media 
technology 

The focus in multi media should be on the concepts of instructional visualisation and on the 
processes of what to use it for like for instance instruction, exploratory learning or searching 
for information. Theories of educational and cognitive psychology in the area of visualisation 
have to be worked out and application domains have to be explored, rather than focussing on 
the more or less autonomous development of technology. 

Focus on integrated and empirical approach rather than on isolated theoretical research 
Research, development and practical application and evaluation should be closely related. 
Empirical oriented projects should be prioritized, in which different variables can be studied in 
practice. One should go for an approach in which reference models and guidelines, derived 
from theory and experience, form the basis of design and development and in which evaluation 
in practice should lead to adapting a research and design and development approach. Such an 
emphasis leads to taking into account economic, organisational, technical and cultural asepcts 
in research and development. This empirical approach might lead to faster progress in this area 
than an isolated approach of studying only certain factors. 

On a more detailed level we come to the following conclusions regarding concepts on 
visualisation, design and development and implementation: 

Broadening the spectrum of applicability 

By using visualisation not only cognitive skills can be developed, but also - to a certain degree 
- more affective skills like learning how to communicate, how to negociate, how to sell. 
Besides, using a rich, visualisation oriented environment also seems to motivate people. One 
could see this increased motivation as a basis for acquiring cognitive and affective skills, but 
one might also see the development of motivation as a goal in itself. This is for instance 
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relevant in business situations, where one wants to communicate for instance the mission and 
vision of the compancy, but with the explicit goal of increasing motivation or incentive. This 
process can go hand in hand with regular development of cognitive and affective skills. So we 
could say that with multi media the spectrum of applicability is broadened. 

Choosing the right technological ambition level 

The final goal of all research and development is that products will be used. Given fast 
changes in technology and the long R&D cycle and process of product implementation, there 
is a danger that reseachers and developers are running behind the technology. Although a 
product might still be quite good, one has to take into account the - often subjective - and 
changing acceptance and tolerance of the users. Think for instance about the shift from black- 
white to colour television. So, together with a strong focus on conceptual and design aspects, 
also a smart strategy has to be followed in terms of production and implementation. One 
should check that products really will be used before they are technologically outdated and 
possibly not being accepted by users anymore. Of course the other point is the danger of 
jumping to fast in new technology. And although research has to be done, one should not 
uncritically embrace technological developments as useful in themselves. For instance, 
hypermedia - making use of the hot spot concept - was there before anyone had really 
experimented and evaluated its value. Also with virtual reality there is the danger of letting 
technological developments lead the way. 

Focus on content design, being of a combined creative and industrial nature 
With multi media, expertise in graphical design and audio-video production becomes 
important. However, this should not be seen as an isolated process. It has to be integrated 
with the interaction oriented design of computer based training. Thic demands for good 
methods and training of people. 

The design process, especially of the combined interaction / multi media presentation modes, 
is also a creative process as well as an industrial process. It is industrial because intermediate 
products, guidelines and standards and roles of people can be structured on beforehand. 
However, within the context of these pre-defined activities and roles creativity is needed. 
Especially making things visible is a creative process. So, design methods should allow and 
encourage people to be creative. New technology in itself does not mean or guarantee more 
creativity on beforehand. Even with textual based systems one can still be creative in order to 
develop an acceptable solution within the given context. 

The use of visualisation in tools: interactivity and integration 
In our definition of multi media we not only have the presentation side of it. We add 
interaction as a relevant dimension. It is the combination of multi media presentation and 
interaction which brings the added value. Interaction may mean for instance two way 
interaction like questioning and answering as in training. But interaction means also a mainly 
one way process of storage, retrieval and navigation as in a point of information. In 
developing tools it is the combination of interactivity and visual support which can lead to 
better quality tools. 

Integration of different tools and applications is becoming more and more common. For 
instance the linking of a spreadsheet (as an object) in a text. However, integration in terms of 
exchanging data, encapsulating objects and using parts of the functionality of another 
application are no easy to understand activities. Using visualisation in order to support these 
integration processes in design- and development might lead to higher quality tools. 
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